Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Flume Hive sink error

Solved Go to solution

Flume Hive sink error

New Contributor

Hi,

I'm trying to ingest data with flume into hive table using appropriate hive sink. The table is partitioned with column date(e.g. "2016/05/26") and clustered into 5 buckets. Also all requirements for hive streaming are specified as described in StreamingRequirements

But all the time I get the same error:

FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook) java.lang.ClassNotFoundException: org.apache.atlas.hive.hook.HiveHook at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:60) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1376) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1360) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1589) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1262) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1126) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116) at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.runDDL(HiveEndPoint.java:456) at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.createPartitionIfNotExists(HiveEndPoint.java:421) at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:280) at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:246) at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoint.java:183) at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:160) at org.apache.flume.sink.hive.HiveWriter$8.call(HiveWriter.java:376) at org.apache.flume.sink.hive.HiveWriter$8.call(HiveWriter.java:373) at org.apache.flume.sink.hive.HiveWriter$11.call(HiveWriter.java:425) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

I use hdp 2.4.2 sandbox with default flume version. How can I resolve this issue?

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Flume Hive sink error

it looks that atlas hook is enabled in your sandbox, could you please check your hive-site.xml change hive.exec.post.hooks as follows if you dont want to use atlas hooks

from

<property>

<name>hive.exec.post.hooks</name>

<value>org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook</value>

</property>

to

<property>

<name>hive.exec.post.hooks</name>

<value>org.apache.hadoop.hive.ql.hooks.ATSHook</value>

</property>
2 REPLIES 2

Re: Flume Hive sink error

it looks that atlas hook is enabled in your sandbox, could you please check your hive-site.xml change hive.exec.post.hooks as follows if you dont want to use atlas hooks

from

<property>

<name>hive.exec.post.hooks</name>

<value>org.apache.hadoop.hive.ql.hooks.ATSHook, org.apache.atlas.hive.hook.HiveHook</value>

</property>

to

<property>

<name>hive.exec.post.hooks</name>

<value>org.apache.hadoop.hive.ql.hooks.ATSHook</value>

</property>

Re: Flume Hive sink error

New Contributor

thanks, that fixed the issue!

Don't have an account?
Coming from Hortonworks? Activate your account here