question Hive queries without from clause seem to fail when Atlas HiveHook is enabled in Support Questions

Hive queries without from clause seem to fail when Atlas HiveHook is enabled

haeusler — Tue, 08 Mar 2016 02:15:21 GMT

When testing Hive queries in HDP 2.4.0 Sandbox, I noticed that simple queries without a from clause fail.

In the Ambari Hive View, they fail with "Error while processing statement: FAILED: Hive Internal Error: java.lang.NullPointerException(null)"

On the command line, they fail with the following stacktrace:

hive> SELECT 42;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:109)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:270)
	at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:321)
	at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:214)
	at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

When removing the "org.apache.atlas.hive.hook.HiveHook" from "hive.exec.post.hooks" this query works:

hive> select 42;
OK
42
Time taken: 6.483 seconds, Fetched: 1 row(s)

Re: Hive queries without from clause seem to fail when Atlas HiveHook is enabled

shwethags — Wed, 09 Mar 2016 14:03:43 GMT

Thats an issue. Created https://issues.apache.org/jira/browse/ATLAS-556 to track this. Thanks for reporting it

Re: Hive queries without from clause seem to fail when Atlas HiveHook is enabled

ibobak — Thu, 07 Apr 2016 18:38:39 GMT

I solved it by editing the /etc/hive/conf/hive-site.xml and changing the line

<value>org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook</value>

onto

<value>org.apache.hadoop.hive.ql.hooks.ATSHook</value>

(thus, removing the unnecessaty HiveHook)

Re: Hive queries without from clause seem to fail when Atlas HiveHook is enabled

visharma — Thu, 28 Apr 2016 18:19:08 GMT

@Michael Häusler

Do you still get the error while running hive queries without a tablename.

I tried reproducing this error with following versions of software:

atlas-0.7

hive-1.2.1

hadoop-2.7.2

The query executed successfully without any exception.

It seems the issue has been fixed in one of the previous patches for Atlas.

Please reply if you still get the error.

Re: Hive queries without from clause seem to fail when Atlas HiveHook is enabled

bigdataam09 — Fri, 27 May 2016 03:06:56 GMT

I tried each and every possible solution mentioned in this forum and in stackoverflow, but it did not resolve my issue.

Finally, I resolved it by copying all the jars in <atlas package>/hook/hive to lib (create a new lib folder at job.properties level) folder of my oozie workflow