In our pipeline we are using PutDatabaseRecord to insert into database. we observed performance bottleneck in PutDatabaseRecord. The flow file is getting queued up onPutDatabaseRecord. Task/Time for PutDatabaseRecord is as follow 167,281 / 01:46:52.678. Total 45 connections configured for DBCP connection pool service.
is it recommended to have MergContent before PutDatabaseRecord to batch flow file together? what will happe if any one of the flow file fails? will the whole batch is getting rolled back? Please let me know.
Your whole flow will perform better if you have flow files with many records, rather than 1 record per flow file.
PutDatabaseRecord will set auto-commit to false on the connection, then start executing the statements, and if any failure happens it will call rollback on the connection.