Member since
06-07-2016
6
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4506 | 06-08-2016 11:46 PM |
06-15-2016
07:07 PM
No problem Rafael, glad it works for you now. It sounds like it was a problem with the networking of the containers that for whatever reason they couldn't establish a connection. Hopefully it doesn't manifest again!
... View more
06-15-2016
03:42 AM
Hi @Rafael Gomez, If the connection is stalling indefinitely, my first inclination would be to check the components and the networking between the two. Is Hive up on running on HDP? Can you connect to it via a client like Beeline from outside the sandbox? Can the Zoomdata host see the Hive server (e.g., verify via Telnet) What do the Zoomdata logs say? There should probably be eventually something to the effect of Connection Timeout or Connection Refused. Please share any relevant stack traces. Thanks!
... View more
06-08-2016
11:46 PM
@Tim Apps and @Joeri Smits It will not work out of the box because HDP 2.4 Sandbox has an Atlas Bridge that's throwing the error on the post Hive hook. This happens for any query involving a "dummy" database (e.g., "select 1"). Via the Ambari config for Hive you should locate the hive.exec.post.hooks property. It will have a value like this: org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook You should remove the second hook (the Atlas hook) so the property only reads: org.apache.hadoop.hive.ql.hooks.ATSHook Lastly, restart the Hive server. You should be able to verify the fix even without Zoomdata by executing in Hive (via Ambari or Beeline): select count(1); If the query is working, Zoomdata should have no problem connecting to HDP's Hive. Hope this helps!
... View more
06-08-2016
11:38 PM
Thanks for the tip. It was set to this: <property>
<name>hive.exec.post.hooks</name>
<value>org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook</value>
</property> Removing the Atlas hook (org.apache.atlas.hive.hook.HiveHook) allowed the query to run unimpeded.
... View more
06-08-2016
07:10 PM
Thanks for the response. I tried disabling Atlas (and only Atlas) in Ambari as admin. Afterward, I tried executing any Hive query through both Ambari or Beeline. In both cases I get an error: Error: Error while processing statement: FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused) (state=08S01,code=12) It seems Hive and Atlas are tightly coupled. Working against any sort of "dummy database" causes an error in the Atlas bridge which propagates up to Hive. Example: 8: jdbc:hive2://localhost:10000> select count(1);
INFO : Tez session hasn't been created yet. Opening session
INFO : Dag name: select count(1)(Stage-1)
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1465411333149_0001)
INFO : Map 1: -/-Reducer 2: 0/1
INFO : Map 1: 0/1Reducer 2: 0/1
INFO : Map 1: 0(+1)/1Reducer 2: 0/1
INFO : Map 1: 1/1Reducer 2: 0(+1)/1
INFO : Map 1: 1/1Reducer 2: 1/1
Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.NullPointerException(null) (state=08S01,code=12)
8: jdbc:hive2://localhost:10000> select 1;
Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.NullPointerException(null) (state=08S01,code=12)
... View more
06-07-2016
06:05 PM
We have a third party product that tries to execute a simple bit of SQL to verify that the user can execute MR/Tez jobs against a source. It attempts to run: SELECT COUNT(1) Which works under a normal Hive installation. However, in the HDP 2.4 sandbox it results in an error. Logs indicate: Query ID = hive_20160607173731_61e36107-5139-477b-a52a-fe20b0ee04b6Total jobs = 1Launching Job 1 out of 1Status: Running (Executing on YARN cluster with App id application_1465317865651_0003)Map 1: -/-Reducer 2: 0/1Map 1: 0/1Reducer 2: 0/1Map 1: 0(+1)/1Reducer 2: 0/1Map 1: 1/1Reducer 2: 0/1Map 1: 1/1Reducer 2: 0(+1)/1Map 1: 1/1Reducer 2: 1/1OKFAILED: Hive Internal Error: java.lang.NullPointerException(null)java.lang.NullPointerExceptionat org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:109)at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:270)at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:321)at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:214)at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:415)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)at java.util.concurrent.FutureTask.run(FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745) Assuming we can't change the source query, can we get a fix?
... View more
Labels:
- Labels:
-
Apache Hive