Curious to know what do folks think about using HDInsight as a production cluster. Requirements: - daily ETL from MSSQL server from customer site into HDInsight - pulling data into Hive - transformations on data -- so far possible though HiveQL and UDF - connection to Tableau ... View more

Folks, I am running a HDInsight cluster over Azure. I'd like to see the executing jobs etc., but when I do YARN -> Quick Links -> Resource Manager UI, Internet Explorer times out with the following error. Make sure the web address http:// activate node>.jx.internal.cloudapp.net:8088 is correct. Look for the page with your search engine. Refresh the page in a few minutes. Check that all network cables are plugged in. Verify that airplane mode is turned off. Make sure your wireless switch is turned on. See if you can connect to mobile broadband. Restart your router. Fix connection problems Same is the case with MapReduce2->Quick Links -> Job History UI. Do I need to enable viewing of these web-pages? ... View more

Folks, My peer has setup HDInsight over Azure with his credentials/resource group. He has shared the admin username and password with me. With this, we are able to login to Ambari using some laptops but not others... While I can login to <mmm>.azurehdinsight.net, I cannot login Ambari using the same credentials. I cleared my Chrome cache, restarted my Mac, tried but no luck. - How come? - Is it because I am running my own <nnn>.azurehdinsight.net at the sametime? - is there anything to do with my browser? ... View more

I am trying to Sqoop data from a local on-site Microsoft SQL server to Hive in HDinsights in Azure. Inside Secure Shell in HDInsight, I CAN sqoop data in from Azure SQL Server but UNABLE to sqoop data from SQL Server locally. I can also setup Azure SQL Server, setup firewall setting and synchronize with local SQL Server. I cannot connect/ping the external SQL Server's ip. - why can't my sqoop connect to the local SQL Server? - are there any firewalls setup in HDInsight that prevents connection? ... View more

Storm can be kerberized in 2.2. Kafka cannot, it needs to be upgraded to 2.3 - Can an external kerberized kafka cluster (non-HDP) pipe data into kerberized Storm present in HDP 2.2 - Is there a workaround to Kerberizing Kafka in HDP 2.2? Thanking in advance ... View more

Folks, What is the recommended value for "yarn.nodemanager.resource.local-dirs"? We only have one value (directory) configured for the above property, which has a size of 200GB. Our hive jobs' map/reduce fill this folder up, and yarn places this node in the blocklist. Moving to tez engine and/or increasing the quota size may fix this, but we'd like to know the recommended value. ... View more