Support Questions
Find answers, ask questions, and share your expertise

Problem to send a file to hive

Hello everyone, I have a problem when I try to send a csv file from nifi to hive. I have this error message that appears that I do not understand and I also have this email that I get that I do not understand too much either. I tried to encode the file in UTF-8 then Cp-1252, there is nothing that works. Would anyone have an idea please?


Super Guru

Can you share the contents of the flow file going into the PutHiveQL processor? From the filename attribute, it appears that the CSV data has been translated into SQL (although you will need to verify that it is valid HiveQL, they are not the same thing). But the error implies the flow file content is still in CSV format, which PutHiveQL will not accept (prior to NiFi 1.2.0 it expects a single HiveQL statement as the content of the flow file).

In general, you would want to split each CSV row into its own flow file (perhaps with SplitText), then translate the CSV data into an INSERT statement (using ReplaceText), then send the statement to PutHiveQL. As of NiFi 1.2.0, PutHiveQL will accept multiple statements (delimited by a semicolon), so you will be able to use a MergeContent processor to combine all the HiveQL statements into a flow file for execution by a single session in PutHiveQL.

<a href="/storage/attachments/15077-flux.png">flux.png</a>Thank you for your reply.
I was already using the same reasoning for my flowfiles. This worked for all my files, ie 95%. But for this one my task does not work and the file does not send. Do you have any idea why?flux-2.png