Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi Rolling Update Strategy in Kubernetes (to avoid downtime)

avatar
Explorer

Hi!

Currently, we're utilizing a 3-node Nifi setup within Kubernetes, incorporating a custom NAR to integrate custom processors (approximately 30 processors developed).

When updating Nifi along with our custom processors, we typically follow these steps:

- Develop a new docker image containing the updated NAR version.
- Modify our Kubernetes deployment to incorporate the new docker image.
- Delete the Kubernetes deployment, resulting in a complete Nifi stop.
- Recreate the Kubernetes deployment, initiating Nifi with the new docker image.

Though effective, this process introduces downtime.

We're contemplating transitioning to a rolling update approach, wherein each Nifi node undergoes a stop-start cycle. However, in Nifi 1.7 at least, the node that is stopped and restarted cannot to join the cluster due to flow differences in the version of our custom processors. We're curious whether this issue persists in the latest Nifi version, say 1.25.

Another option involves deploying multiple versions of the NARs in Nifi. However, changing processor versions manually, especially considering the abundance of custom processor instances we have, presents a challenge. Ideally, in this scenario, upon restarting the pod, we'd like Nifi to automatically opt for the latest version to obviate manual intervention.
Another issue to comment is that we have a Controller Service that many of our processors use and to change the version of the listener we need to stop all processors attached to it.

So to summarize:

a) If we use only one version of the NAR and we use a rolling udpate strategy, in the last version of Nifi we will continue having the problem that a pod cannot join the cluster if it have an updated version of the processor?

b) If we use multiple NARs adn a rolling update strategy. Is there any way when we restart a nifi pod to update all our custom processors and listeners to switch to the last available version?

Thanks for any suggestions or advice!!!

Best Regards,
JP

 

 

 

 

 

 

 

 

 

2 ACCEPTED SOLUTIONS

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

Thanks a lot for the information RAGHUY and MathWho for the comments. I will accept the solution. We have a lot of integrations built on nifi 1.X so I think we will try to find a way to overcome the problem at least for our versions and then migrate to 2.X when possible.

Thanks again!!