Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

tar size limit in MergeContent ?

avatar
Frequent Visitor

Is there a size limit to the tar file generated by MergeContent?

ERROR

MergeContent[id=8a06a034-0183-1000-b8b5-c045959a56e9] Failed to process bundle of 2 files due to entry size '14743306287' is too big ( > 8589934591 ).; rolling back sessions: java.lang.RuntimeException: entry size '14743306287' is too big ( > 8589934591 ).

 

JohnMaxQ_0-1664553106316.png

 

 

1 ACCEPTED SOLUTION

avatar
Master Mentor

@John-MaxQ 

The MergeContent processor utilizes the Apache Commons Compress library which has a hard limit in tar size.  There is an existing Apache NiFi jira for this here:
https://issues.apache.org/jira/browse/NIFI-10273

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt



View solution in original post

1 REPLY 1

avatar
Master Mentor

@John-MaxQ 

The MergeContent processor utilizes the Apache Commons Compress library which has a hard limit in tar size.  There is an existing Apache NiFi jira for this here:
https://issues.apache.org/jira/browse/NIFI-10273

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt