how to use puthivestreaming

121285904 — Sun, 02 Oct 2016 21:09:35 GMT

I want to move data from hdfs to hive using puthivestreaming of nifi. does anyone have example?

Re: how to use puthivestreaming

mkalyanpur — Mon, 19 Aug 2019 11:39:58 GMT

Hive streaming is supported against tables having the following :

ORC is the only format supported currently. So your table must have "stored as orc"
transactional = "true" should be set in the table create statement
Bucketed but not sorted. So your table must have "clustered by (colName) into (n) buckets"

Also, Hive must have the following properties set

PutHiveStreaming requires your input data to be in Avro format, as is provided in the documentation here:

Considering your data is JSON, we can construct a Nifi workflow as follows:

ListHDFS--> FetchHDFS--> ConvertJsonToAvro-->PutHiveStreaming

Let me know if this helps.

mburgess — Tue, 04 Oct 2016 20:08:53 GMT