Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi GetKafka how are offsets stored in Nifi? Kafka & Nifi integration?

Nifi GetKafka how are offsets stored in Nifi? Kafka & Nifi integration?

Contributor

If a kafka node dies in the kafka cluster how will Nifi handle it?

If the whole cluster is down for some reason how does nifi deal with it? How will nifi know how many offsets it has read so far? Does nifi store offsets in memory?

If nifi is down it needs offsets not to read duplicate messages from kafka? How can we make nifi fault tolerant as we are planning to productionize it?

Is there any detailed documentation about integration of nifi and kafka?

Which alerting or monitoring system is recommended to use with Nifi in production?

Can i use getKafka as a batch rather than streaming like schedule my nifi flow to be once or twice a day consume all the messages of that day from nifi probably an overnight job to consume all the new events produced to kafka on that day?

At the moment we have a 4 node nifi cluster with embedded zookeeper? Do i use my embedded ZK to store offsets or how can i configure nifi to store or use offsets from kafka ZK?

What will Nifi do if a leadership election is invoked while its consuming?

What happens if i restart nifi and lot of messages were produced in kafka. How will nifi read them and how can it figure out the new message offfsets?

Don't have an account?
Coming from Hortonworks? Activate your account here