Hi @kavitha velaga for this kind of monitoring, I'd suggest using an external monitoring framework, something like Munin, Ganglia or whatever framework you already use within your org. Most of these frameworks can handle the recording of Round Trip Times (RTT) from hosts to something like an s3 endpoint. Hope that helps. ... View more

Hi @Anandha L Ranganathan All the steps are covered in the ambari wizard for the downgrade, but you'll need to pay close attention to the various databases etc that require backing up as you go through. Both roll-back and full downgrade are possible. Make sure you have satisfied all of the prerequisites before beginning as well, and preferably have read through the upgrade document several times to ensure you have accounted for everything. http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_upgrading_Ambari/content/_upgrading_HDP_prerequisites.html I've performed a number of upgrades myself and while there is always the odd thing that crops up, its usually something minor that is easily fixed. Good luck and I hope everything goes smoothly. ... View more

Hi @ccasano There was a post that covers the minimum privilidge set that you require for Cloudbreak on AWS by @lpapp. https://community.hortonworks.com/questions/30242/list-of-policies-required-by-cloudbreak-to-launch.html As for the intracacies of VPC management, unless anyone here knows, that might be a question better answered by Amazon. ... View more

Hi @Fazil Aijaz yes I believe those issues have been resolved, as you noted, a good fast internet connection is certainly required, but I know a few people who have completed the exam sucessfully in the last week or so. I will verify with the team that everything is working as expected, but if you don't hear from me here in the next day or so then everything is good to go. Good luck with your exam! ... View more

Hi @John Yawney. So answering your questions one at a time: 1) Currently Atlas 0.6 (that or later is expected in the next HDP release) supports the following hooks (therefore tracks the govornanace information for data that is touched by these systems) Hive, Sqoop, Falcon, Storm (with Spark, NiFi and HBase expected around end of year). Currently anything that doesn't have a hook won't be tracked. For time frame information on the next release, there's no public information but if you look historically we have regularly announced new releases around the US Hadoop Summit timeframe. This gives a good idea of what is coming down the line for Atlas: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62695330 2) Atlas is designed to be open and extensible so you could absolutely add 3rd party liniage information into the metastore. Any work you do in that area would also be greatly appreciated if contributed upstream also, with the added benifit that when accepted you won't need to suppor that code on your own. 3) Assuming you're talking Hive here? Column level liniage is expected around the end of the year. 4) You can, usually in combination with Ranger, but you'll need to be more specific about exactly what you mean for these rules. Additional notes.... For more detailed information, as I know the documentation around Atlas is pretty poor... I'd strongly advise watching three sessions that occurred during the recent European Hadoop Summit, search for sessions by Andrew Ahn (there are three!) http://www.hadoopsummit.org/dublin/agenda/ ... View more

Hi @Sunile Manjee. It's a bit more fully featured than that, Wandisco solutions generally operate at a level above the underlying service, so if one of the HBase clusters dies, the users don't notice any change, the second area of significant importance is Wandisco's solution ability to handle significant distances between clusters, where as standard HBase replication is usually within the same datacentre or only via very fast links, not full geo replication. https://www.wandisco.com/product/fusion-active-active-hbase They also have a webinar on HBase, the final section is all about using Fusion Active HBase across multiple datacentres: http://www.wandisco.com/webinar/replay/hadoop-hbase-depth Hope that helps! ... View more