Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

putHive3Streaming Error

putHive3Streaming Error

Hello,

I have created this table to test puthive33streaming processor :

create table dev.streaming_orc 
(
 id int,
 text string
)
CLUSTERED BY (id) INTO 5 BUCKETS
stored as ORC
TBLPROPERTIES ('transactional'='true');

I got the following error message :

2018-10-25 11:07:38,152 ERROR [Timer-Driven Process Thread-6] o.a.h.streaming.HiveStreamingConnection Fatal error on TxnId/WriteIds=[14839/1...14839/1] on connection = { metaStoreUri: thrift://xxx:9083,thrift://xxx:9083, database: dev, table: streaming_orc };  TxnStatus[O] LastUsed txnid:0; cause nulljava.lang.NullPointerException: null  at org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)  at org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)  at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)  at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)  at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)  at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.<init>(HiveStreamingConnection.java:677)  at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.<init>(HiveStreamingConnection.java:596)  at org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)  at org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)  at org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)  at org.apache.nifi.processors.hive.PutHive3Streaming.onTrigger(PutHive3Streaming.java:419)  at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)  at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)  at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)  at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)

It is like it is a problem of metastore initialisation ??

Can anybody know where the error come from ?

Thx in advance

1 REPLY 1
Highlighted

Re: putHive3Streaming Error

Solved !

I needed to add /etc/hive/conf/hive-site.xml,/etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml to hive configuration resources tab.

And give nifi write access to folder in /warehouse/tablespace/managed/hive/

Don't have an account?
Coming from Hortonworks? Activate your account here