Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

issue with "Puthivestreaming" Processor version 1.4 - Using a stand alone Version of Nifi.

Expert Contributor

Hi All ,

Need some help here with the "PutHiveStreaming" Processor. I have followed all the prerequisites to set up the data flow onto "PutHiveStreaming" processor however I continue to get the following error :-

Hive Streaming connect/write error, flow file will be penalized and routed to retry. org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint

I have seen this is a fairly common error across the Nifi Forums and I guess @Matt Burgess had posted a NAR file to fix this issue however the NAR file was of version 1.0 and in my case wont work as I am using version 1.4 which is a stand alone open source version of Apache Nifi ( not HDP ). Our requirement is to extract data from database and stream it into a partitioned table on Hive. We have used the streaming option to write onto HDFS using QueryDatabaseTable -> ConvertAvrotoOrc --> putHDFS however this would just land files onto HDFS , we need to directly insert data onto the partitioned table and make data available to the end users in real time.

@Matt Burgess

@Shu

Do you have any idea on how to get this processor working by any chance

1 ACCEPTED SOLUTION

Expert Contributor

Ok , we just got this fixed , It seemed to be some issue with the thrift server on Hive. all good 🙂

View solution in original post

3 REPLIES 3

Expert Contributor

Ok , we just got this fixed , It seemed to be some issue with the thrift server on Hive. all good 🙂

Abhinav,

I am seeing the same error, How did you fix this?

Expert Contributor

Hi @Srinatha Anantharaman - we actually got in touch with our Hadoop admin and opened all firewalls for the thrift metastore server after which it worked, Hope this helps

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.