Unable to load CSV data in to Hive table using Puthivestreaming using Nifi ?

[#1]

Below are the errors i'm getting while loading CSV file to HIVE table using NIFI.

1.Hive streaming connect/write error, flowfile will be penalized and routed to retry.

2.Error connecting to Hive endpoint:table routes at thrift.

3.Failed to create hive writer for endpoint.

Any solutions for the above much appreciated& Thanks a lot in advance.

[#2]

Bhanu, can you please share the entry of the nifi-app.log for the error to see what exactly happened?

[#3]

please check this, this helped me

https://community.hortonworks.com/questions/59681/puthivestreaming-nifi-processor-various-errors.htm...

[#4 · Accepted solution]

Ah, Hive has reached the maximum number of open transactions. There is the parameter hive.max.open.txns that limits the number (compare to https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions ). Default Value is 100000 - this should be high enough, and I guess you did not change it, but maybe you want to check it.

Not sure what is going on in your environment but maybe you want to have a look Chapter 6 of http://hortonworks.com/hadoop-tutorial/using-hive-acid-transactions-insert-update-delete-data/ to see how to manage your open transactions.

Hope this helps.

[#5]

Bhanu @ Please find my nifi_app.log as an attachment.

[#6]

Thanks Bhanu, the error comes from Hive. Can you please also have a look at the hive metastore log file and share any related error message?

[#7]

Bhanu@please find hivemetastore as an attachment.

[#8]

I have changed Open txns limit & enabled Run compactor to ' TRUE' & changed the numbers of threads used by compactor ='5'.Now i'm able to load data in to my Hive table. Thanks a lot for help Peter.It worked like an charm.

[#9]

Very good, glad to help. I transformed the comment that helped finally to an answer and would be happy if you accept it 😉 Thanks.

[#10]

Sure & Thank you very much indeed.