Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Merge big Json using Wait/Notify in Apache Nifi

Highlighted

Re: Merge big Json using Wait/Notify in Apache Nifi

Master Guru

@3nomis

MergeContent should be using Defragment.
There is no default value for Max Bin Age, so not sure what you set there. If left blank, processor will wait for ever to merge a bin unless you run out of bins.
Also make sure you adjust the object and size thresholds on the connections feeding the MergeContent processors so that they are large enough to accommodate the number of splits that need to be merged.
Considering the size of the FlowFiles being merged, it may take time to merge all of them.
as far as bins, try setting 21 of them.


Thanks,

Matt

Don't have an account?
Coming from Hortonworks? Activate your account here