Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

MergeRecord after fragmented ExecuteSQLRecord throwing error

Highlighted

MergeRecord after fragmented ExecuteSQLRecord throwing error

New Contributor

I am trying to query a database and route its results into a CSV file. I am using the following flow:

Nifi Version - 1.8.0

ExecuteSQLRecord > MergeRecord > PutFile

when done as intended, I am seeing the following issue in nifiapp.log

2019-02-04 18:08:32,193 ERROR [Timer-Driven Process Thread-15] o.a.nifi.processors.standard.MergeRecord MergeRecord[id=3152151f-1817-11cb-380b-cbb5c84cefa0] Could not merge bin with 1 FlowFiles because the 'fragment.count' attribute had a value of '5' but only 1 of 5 FlowFiles were encountered before this bin was evicted (due to to Max Bin Age being reached or due to the Maximum Number of Bins being exceeded).

My processes look like following:

ExecuteSQLRecord:

103388-executesqlrecord.png

CSVWriterController:

103385-csvwritercontroller.png

CSVReaderController:

103386-csvreadercontroller.png

MergeRecord:

103387-mergerecord.png

Appreciate all the help.

I did try moving the MergeRecord to use Bin-Packing with Common attribute as filename but that resulted in no-merge happening.

Thank you

103383-executesqlrecord.png