Created 05-12-2017 07:32 AM
Created 05-13-2017 02:33 AM
As of NiFi 1.2.0, after GetFile you can use the PutDatabaseRecord processor with a CSVReader, giving it a schema if you know the layout, or if the CSV file has a header row you can get the column names from that using "Use String Fields From Header" for the Schema Access Strategy property.
Prior to NiFi 1.2.0, after GetFile you probably want a SplitText to get each row in its own flow file. Do you know the number of columns for the CSV file? If so then you can use a regex in ExtractText; assuming there were four columns you might have a dynamic property called "column" set to something like:
([^,]+),([^,]+),([^,]+),([^,]+)
That should give you attributes "column.1", "column.2", "column.3", and "column.4". Then you can use ReplaceText to generate a SQL statement, perhaps something like:
INSERT INTO myTable VALUES(${column.1},${column.2},${column.3},${column.4})
Then send that to PutSQL. Another option (prior to NiFi 1.2.0) is to convert the CSV to Avro (see this HCC article), then ConvertAvroToJSON, then ConvertJSONToSQL, then PutSQL.
Created 05-13-2017 02:33 AM
As of NiFi 1.2.0, after GetFile you can use the PutDatabaseRecord processor with a CSVReader, giving it a schema if you know the layout, or if the CSV file has a header row you can get the column names from that using "Use String Fields From Header" for the Schema Access Strategy property.
Prior to NiFi 1.2.0, after GetFile you probably want a SplitText to get each row in its own flow file. Do you know the number of columns for the CSV file? If so then you can use a regex in ExtractText; assuming there were four columns you might have a dynamic property called "column" set to something like:
([^,]+),([^,]+),([^,]+),([^,]+)
That should give you attributes "column.1", "column.2", "column.3", and "column.4". Then you can use ReplaceText to generate a SQL statement, perhaps something like:
INSERT INTO myTable VALUES(${column.1},${column.2},${column.3},${column.4})
Then send that to PutSQL. Another option (prior to NiFi 1.2.0) is to convert the CSV to Avro (see this HCC article), then ConvertAvroToJSON, then ConvertJSONToSQL, then PutSQL.
Created 05-13-2017 03:29 PM
thanks for response...
i have hard coded in replacetext processor (for testing purpose) such as: insert into table_name (ID) values ('2') and in putsql processor i am getting same insert statement(in data provenance input claim view). and same table table_name i have create in data base but i am not getting any error and it's not going to insert.. connection with data base table is fine(no issue)..
flow is getfile>>replacetext>>putsql.
Created 10-08-2019 03:25 AM
Moved because this question differed significantly from the original thread and the original question was marked 'solved' in May 2017.