Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive queries without from clause seem to fail when Atlas HiveHook is enabled

avatar
Contributor

When testing Hive queries in HDP 2.4.0 Sandbox, I noticed that simple queries without a from clause fail.

In the Ambari Hive View, they fail with "Error while processing statement: FAILED: Hive Internal Error: java.lang.NullPointerException(null)"

On the command line, they fail with the following stacktrace:

hive> SELECT 42;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:109)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:270)
	at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:321)
	at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:214)
	at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

When removing the "org.apache.atlas.hive.hook.HiveHook" from "hive.exec.post.hooks" this query works:

hive> select 42;
OK
42
Time taken: 6.483 seconds, Fetched: 1 row(s)
1 ACCEPTED SOLUTION

avatar
Cloudera Employee

Thats an issue. Created https://issues.apache.org/jira/browse/ATLAS-556 to track this. Thanks for reporting it

View solution in original post

4 REPLIES 4

avatar
Cloudera Employee

Thats an issue. Created https://issues.apache.org/jira/browse/ATLAS-556 to track this. Thanks for reporting it

avatar
New Contributor

I solved it by editing the /etc/hive/conf/hive-site.xml and changing the line

<value>org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook</value>

onto

<value>org.apache.hadoop.hive.ql.hooks.ATSHook</value>

(thus, removing the unnecessaty HiveHook)

avatar
Explorer
@Michael Häusler

Do you still get the error while running hive queries without a tablename.

I tried reproducing this error with following versions of software:

atlas-0.7

hive-1.2.1

hadoop-2.7.2

The query executed successfully without any exception.

It seems the issue has been fixed in one of the previous patches for Atlas.

Please reply if you still get the error.

avatar
Contributor

I tried each and every possible solution mentioned in this forum and in stackoverflow, but it did not resolve my issue.

Finally, I resolved it by copying all the jars in <atlas package>/hook/hive to lib (create a new lib folder at job.properties level) folder of my oozie workflow