Even, I cant find how one can get transactionId.. Any help greatly appreciated! ... View more

Ah! James Jones --- I have a Recommendation - Never believe recommendations from Ambari 😉 ... View more

I had a similar problem with pyhive on my horton setup. The failure was always immediate and so it was not a timeout issue that some people on the net were pointing out. It turned out to be hive.server2.transport.mode. If this is set to binary, it works like a Charm. If it is 'http', PyHive does not work. Also found https://github.com/dropbox/PyHive/issues/69 which talks about this. HTH. ... View more

We had this issue... What I found from logs is that the livy client copies this zip from /tmp/sparkxxxxx to HDFS:/user/<username>/.sparkStaging/.... In our case, livy user impersonation was enabled. Hence <username> Livy wad able to create ".sparkStaging/<application-id>" directory... but that directory is owned by the <username> with 700 permission.... And then the zip file could not be copied to that directory by Livy. I found this by a writing a small shell script that continuously spins "ls -lR"ing the HDFS /user/<username>/.sparkStaging directory.. (while true:; do; hdfs dfs -ls -R /user/<username>/.sparkStaging; done;) --> Ran this as "livy" user.. So, we just gave blanket permissions in Ranger for "livy" to "/user" recursively. And that was it! We got it working! ... View more

Oh yeah! We too hit this problem on our Sandbox cluster running HDP 2.5.3 with Kerberos + AD integrated secure cluster. We can see that Ambari server is not sending the commands to Ambari agent (or) is hung on something with Ambari agent... Just the gear icon in the settings with 3 small icons never appear.... Man... Such a pain, this one is. Today we restarted the services and it took 2 hours for a 4-node cluster.... Hadoop means Elephant, right? Now, I believe it.... for a totally different reason though.... 🙂 ... View more

A learning I had was that when you export a table, use complete HDFS URI... In some cases, I found that it helped executing the command which otherwise was failed ... View more

@Rakesh Gupta Thank you very much. Smart debug sir. Saved me my day today. ... View more

What do you mean by "added host entries of the unsecured cluster in the secured cluster"?? Thats kind of weird. I thought we only add Hosts so that they can be a part of the current cluster.... If you did not add the unsecure host, what problem did you encounter? ... View more

I found the answer to this myself. When you search for ID, it does not necessarily mean "Processor ID". In my case, it was the "Controller Service" ID which was present inside the Processor. Thats a good learning for the day. Thanks much. ... View more