Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Solved Go to solution

issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Expert Contributor

Hi All ,

Need some help here with the "PutHiveStreaming" Processor. I have followed all the prerequisites to set up the data flow onto "PutHiveStreaming" processor however I continue to get the following error :-

Hive Streaming connect/write error, flow file will be penalized and routed to retry. org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint

I have seen this is a fairly common error across the Nifi Forums and I guess @Matt Burgess had posted a NAR file to fix this issue however the NAR file was of version 1.0 and in my case wont work as I am using version 1.4 which is a stand alone open source version of Apache Nifi ( not HDP ). Our requirement is to extract data from database and stream it into a partitioned table on Hive. We have used the streaming option to write onto HDFS using QueryDatabaseTable -> ConvertAvrotoOrc --> putHDFS however this would just land files onto HDFS , we need to directly insert data onto the partitioned table and make data available to the end users in real time.

@Matt Burgess

@Shu

Do you have any idea on how to get this processor working by any chance

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Expert Contributor

Ok , we just got this fixed , It seemed to be some issue with the thrift server on Hive. all good :-)

3 REPLIES 3
Highlighted

Re: issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Expert Contributor

Ok , we just got this fixed , It seemed to be some issue with the thrift server on Hive. all good :-)

Re: issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

New Contributor

Abhinav,

I am seeing the same error, How did you fix this?

Re: issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Expert Contributor

Hi @Srinatha Anantharaman - we actually got in touch with our Hadoop admin and opened all firewalls for the thrift metastore server after which it worked, Hope this helps