Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

best practices - Integrate Nifi with existing HDP

avatar
Contributor

We have an existing HDP cluster with the typical infrastructure setup: hive/spark/oozie/ranger (all kerberized). What is the best way to integrate a nifi cluster to this cluster? Keeping in mind upgrading and easy existing integration with security etc.

1 ACCEPTED SOLUTION

avatar
Super Mentor
@Sonny Heer

The HDP stack does not include NiFi. HDF is an entirely different software stack managed by its own Ambari and is installed on its own hardware. There are some post out there that will walk you through manually adding the NiFi service to an existing HDP stack; however, doing so breaks the upgradeability of your HDP stack.

Here is an example of that:

https://github.com/abajwa-hw/ambari-nifi-service

*** Pay special attention to the "Limitations" noted on the above link.

At some point in the future there will be an Ambari platform that can be used to install both HDP and HDF stack services that will still support the upgradeability, but that is not here yet and I don't know hen that will happen.

Thank you,

Matt

View solution in original post

2 REPLIES 2

avatar
Super Mentor
@Sonny Heer

The HDP stack does not include NiFi. HDF is an entirely different software stack managed by its own Ambari and is installed on its own hardware. There are some post out there that will walk you through manually adding the NiFi service to an existing HDP stack; however, doing so breaks the upgradeability of your HDP stack.

Here is an example of that:

https://github.com/abajwa-hw/ambari-nifi-service

*** Pay special attention to the "Limitations" noted on the above link.

At some point in the future there will be an Ambari platform that can be used to install both HDP and HDF stack services that will still support the upgradeability, but that is not here yet and I don't know hen that will happen.

Thank you,

Matt

avatar
Contributor

Thanks @Matt Clarke

I understand HDF is separate. I'm looking for the best approach to use existing HDP with HDF. I'll look into ambari-nifi-service again - i didn't pursue this service because of the disclaimer about it being for demo purposes. Anyone using this service - is it up to date with 2.4? Assuming i don't care about upgradability at the moment...

Prefer to add nifi cluster to existing HDP kerberized cluster. any ideas/posts/links on how to best do that would be helpful. Is the best approach to setup a whole new ambari system? I would like to avoid doing this. Thanks