Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi-1.0.0 GenerateFlowFile processor creates duplicates when coordinator and primary node are different

NiFi-1.0.0 GenerateFlowFile processor creates duplicates when coordinator and primary node are different

Rising Star

Today I noticed in NiFi-1.0.0 that all of my GenerateFlowFile processors that were set to run on primary node were generating flow files twice instead of only the primary node generating 1 flow file. After a lot of verifying I noticed that in the cluster the coordinator node and primary node were different and not the same node. Till previously whenever zookeeper elected the coordinator and primary node, one node had always been both the primary and coordinator node. So I restarted all the nodes in the cluster but one so that the one remaining would be both the primary and coordinator and that seemed to fix the issue. Because of this I am wondering if it is possible that NiFi-1.0.0 counts the primary and coordinator as two primary nodes when they are different? Has anyone encountered this before? Also I know that NiFi-1.1.0 came out last week so has this issue been fixed there?

3 REPLIES 3

Re: NiFi-1.0.0 GenerateFlowFile processor creates duplicates when coordinator and primary node are different

Hello this is not a known issue that I am aware of. I have tried to reproduce it and I cannot. Can you provide more details on what you did and how you know FlowFiles were generated by a "Primary Node only" scheduled processor on both the Primary node and coordinator?

Highlighted

Re: NiFi-1.0.0 GenerateFlowFile processor creates duplicates when coordinator and primary node are different

Rising Star

Hi! We are experiencing the same issue today. But in our case the coordinator is the primary node too... Thus, this is not the reason at least in our case. Moreover, our NiFi version is 1.1.0, so the issue has not been fixed. Our processors are CRON scheduled with primary node only as executor, and both the flow files are generated by our primary node. The strange thing is that until yesterday this issue never came out for us. I'm investigating, but as of now the only idea about the reason of this weird behavior is that NiFi doesn't like 2017... ;) Meanwhile, have you found out something more?

Re: NiFi-1.0.0 GenerateFlowFile processor creates duplicates when coordinator and primary node are different

Hello, I believe this issue is different than the one you logged in the Apache Jira. This issue is specific to a cluster and doesn't mention CRON jobs. Your issue I believe is already known and I've marked your ticket as a duplicate: https://issues.apache.org/jira/browse/NIFI-3271

Don't have an account?
Coming from Hortonworks? Activate your account here