- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
PutDatabaseRecord Performance Issue
- Labels:
-
Apache NiFi
Created ‎10-16-2018 06:47 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In our pipeline we are using PutDatabaseRecord to insert into database. we observed performance bottleneck in PutDatabaseRecord. The flow file is getting queued up onPutDatabaseRecord. Task/Time for PutDatabaseRecord is as follow 167,281 / 01:46:52.678. Total 45 connections configured for DBCP connection pool service.
is it recommended to have MergContent before PutDatabaseRecord to batch flow file together? what will happe if any one of the flow file fails? will the whole batch is getting rolled back? Please let me know.
Thanks
Subbu
Created ‎10-17-2018 01:45 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Your whole flow will perform better if you have flow files with many records, rather than 1 record per flow file.
PutDatabaseRecord will set auto-commit to false on the connection, then start executing the statements, and if any failure happens it will call rollback on the connection.
