I want to insert data into the database without converting data into json

[#1]

Hi....I have data in various kinds of files and i want to insert them into database but i don't want it to convert them first into JSON and then to SQL as it is very time consuming.Is there any way so that i can put my data into database from different kinds of files like CSV,Avro etc directly?

[#2 · Accepted solution]

For CSV files, if you know the number and type of column values, you can use SplitText (to get one row per flow file) followed by ExtractText, supplying a regular expression to get the column values out into flow file attributes. Then you can use ReplaceText to manually enter a SQL INSERT statement (using NiFi Expression Language to access the attributes). For other formats like Avro, as we don't currently have a ConvertAvroToSQL processor, you'd have to convert them for now.

Work is underway for a generic system of type conversions, such that you could specify Avro as your input format and perhaps "SQL INSERT" as your output format, thereby effectively making the generic processor work like a ConvertAvroToSQL processor.