Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there any way to distribute the flow.xml.gz in multiple parts?

Highlighted

Is there any way to distribute the flow.xml.gz in multiple parts?

New Contributor

Hello,
My problem is that i have a cluster having thousands of flows running inside it and whenever a change is occurred in any of the flow, doesn't matter how big or small it is, then a call is hit which makes a whole object of the the flow, converts it and saves it into the flow.xml.gz and this same copy of file is then saved to the other nodes.This is taking a very long time, making a save call a lengthy process.
So is there any way, to reduce this time and makes it faster? I am thinking of making a new empty copy of flow.xml.gz file every time i make a save call to a flow, saving only that current flow object for which the call has been made in that empty copy of flow.xml.gz and then merge it to the original flow.xml.gz in the background later in every 10-15 mins which will make my saving process faster. Is it possible or am i going to a wrong way?

Please suggest me the solution.

Thanks in advance

Don't have an account?
Coming from Hortonworks? Activate your account here