Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi cluster :How to configure cluster two process the data on all available nodes means half data by one node and another half data by another node like

Highlighted

Nifi cluster :How to configure cluster two process the data on all available nodes means half data by one node and another half data by another node like

Explorer

We have a nifi two node cluster on vm ,and need to process the data to cosmosDb graph api from azure sql server,i have written a custom processors to get the data from sql server and import into graph api, I have 6 connection stirngs for each user that means 6 flowfiles :

Flow like:first populate the users,items,and items they have bought into graphDb ,like this ,everything is fine fetching data but problem is while inserting into graph db ,each flowfile size 12mb to 30 mb data .

even we have 2 node cluster ,but processing the data only in one node not processing by both ,so its taking time and hanging and getting heap out of memory exception ,not able to insert all users data

Could you plz explain the cluster how to configure two process the data by two nodes .

1 REPLY 1

Re: Nifi cluster :How to configure cluster two process the data on all available nodes means half data by one node and another half data by another node like

Contributor

@siva karna It appears that your processor is configured to run only on 'primary node'. Please check the scheduling tab in the processor configuration, and if it is configured to run only on 'Primary Node' configure it to run on 'All nodes'. Check screenshot.

16698-scheduler.png

Don't have an account?
Coming from Hortonworks? Activate your account here