Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Batch Insert into a Database using NiFi - Invoking UDFs

Batch Insert into a Database using NiFi - Invoking UDFs

New Contributor

Hi All,

My requirement is to load a csv file into a database using NiFi. The current approach (which is inefficient) is SplitText--> ExtractText-->UpdateAttribute-->ReplaceText-->PutSQL. I am looking for a solution that lets me do a batch commit and not individual INSERTs for each FlowFile.

I have seen a couple of responses online like - use PutDatabaseRecord Processor or ConvertJSONToSQL Processor.

But, I have to invoke UDFs in the Insert query to transform the data before inserting into the table. I do not think I will be able to use the above mentioned approaches if UDFs are involved.

Is there a way to Begin and commit the Insert Statements before passing them to PutSQL ? Can we invoke any SQL utility to achieve that? I would really appreciate your help.

Don't have an account?
Coming from Hortonworks? Activate your account here