Merge Content Fragmentation Error

[#1]

I am getting this error:

Cannot Defragment FlowFiles with Fragment Identifier 9718bd03-8285-4a19-9370-444185fd1efb because the expected number of fragments is 0 but found 1 fragments for this identifier; routing [StandardFlowFileRecord[uuid=69d5315d-57bf-4654-8e16-b15e8b9f3e80,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1505745100945-1391548, container=default, section=956], offset=33061, length=24],offset=0,name=3626168233745552.csv,size=24]] to failure

What exactly is this error telling me?

Is this an issue with my source data or is it more likely there is something wrong with my flow?

[#2]

@jacob oatman

This sounds like an issue with your source FlowFile(s).

The "Defragment" requires all FlowFiles to have the following attributes set on them:

The MergeContent processors are configured with virtual "bins". When using "Defragment", FlowFiles from the incoming queue(s) are binned based on the value assigned to fragment.identifier.

The bin is marked complete and ready for merging when the number of FlowFiles in the bin matches the value set in the :fragment.count FlowFile Attribute.

The ERROR above seems to be reporting that a FlowFile reported the fragment.count as 0 and a fragment.count as 1.

How were your source FlowFiles generated?

Have you inspected that the above attributes exist on all your FlowFiles feeding this processor and that they have the correct values set on them?

Thanks,

Matt

[#3]

Hi,

I have the same issue, but it's wanted because i made some lookups before the merge and records that didn't match where skipped, this is wanted behavior!

so the question is, can these "incomplete" flow-files be merged anyway ? how?

Thanks