question Re: Can't enable cluster option for NIFI container in Support Questions

Can't enable cluster option for NIFI container

ZNFY — Tue, 19 Nov 2024 12:17:43 GMT

Hi，

I have now deployed NiFi on my Kubernetes cluster, and for ease of learning, I modified the contents of the start.sh script in the scripts folder to enable HTTP access. I set all certificate-related content to be empty.

Now, I am encountering an issue: when I try to change the environment variable "NIFI_CLUSTER_IS_NODE" to "true", my container fails to start. Strangely, there are no errors in the logs.

When such content appears in the logs, the container stops and attempts to restart, repeatedly going through this restart loop.

It looks all right, doesn't it? But after this the container just stops.

I am wondering if this is because I enabled HTTP access?

By the way, can I start a NiFi cluster using the embedded ZooKeeper? I didn’t see an option to configure "nifi.state.management.embedded.zookeeper.start" in the environment variables of the NiFi docker image.

I hope to get your help, thank you very much！

Re: Can't enable cluster option for NIFI container

MattWho — Tue, 19 Nov 2024 15:00:39 GMT

@ZNFY

You may find this other community thread useful for setting up a Docker NiFi cluster:
https://community.cloudera.com/t5/Community-Articles/NiFi-cluster-sandbox-on-Docker/ta-p/346271

The Apache NiFi Docker is built around setting up a standalone (non-clustered) instance of NiFi.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: Can't enable cluster option for NIFI container

ZNFY — Wed, 20 Nov 2024 05:01:25 GMT

Thank you for your reply. I just tried deploying a NiFi container without modifying the start.sh script, and I found that when I set NIFI_CLUSTER_IS_NODE to true, the container still gets stuck and then stops. I find this strange—if the container doesn’t support clustering, why does the official image allow this environment variable to be set? I think this is misleading.

I noticed that version 2.0.0 supports setting nifi.cluster.leader.election.implementation to KubernetesLeaderElectionManager. Does this require NiFi to be deployed at the same level as Kubernetes, or does NiFi need to be deployed as a pod within Kubernetes? I look forward to your response. Thank you!