Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

From CSV to Hive via NiFi

avatar
Contributor

I want to use NiFi to ingest CSV file (getFile) to Hive (PutHiveQL).

Since HiveQL need the SQL statement, how can I generate it ?

A solution would be getFile -> InferAvroSchema -> ConvertCSVToAvro -> ConvertAvroToJSon -> ConvertJSONtoSQL -> PutHiveQL.

This looks so complex and resource consuming. Any suggestions ?

1 ACCEPTED SOLUTION

avatar
Master Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
11 REPLIES 11

avatar
Expert Contributor

There is no example in the "Working_With_CSV" template of how to extract each individual field into attributes.

avatar

is this solution is fit in streaming more than puthive3ql  for about 10 GB during the day???