Install apache NIFI with ambari on existing HDP cluster

[#1]

Please how to install apache NIFI with ambari on existing HDP cluster

[#2 · Accepted solution]

@Mourad Chahri

If you are looking to demo just NiFi on an HDP cluster, then you can use this: https://github.com/abajwa-hw/ambari-nifi-service

It is unsupported and not intended for production uses. If you want to deploy Hortonworks Data Flow which includes NiFi, then you currently have to install it using a separate version of Ambari from the HDP Ambari.

[#3]

@Michael Young how to install HDF on the same cluster , because i wanna use HDF and HDP

[#4]

@Mourad Chahri

You can not currently install HDP and HDF on the same cluster because there are conflicts with Ambari and the service configurations. You must install HDF and HDP on different servers.

[#5]

@Michael Young can i use the same cluster but , install ambari server fo hdp on a server and ambari-server for HDF on another server but use the same node ??

[#6]

@Mourad Chahri

That is not currently a supported configuration. There are common components in HDF and HDP (Kafka, Storm, Ranger, to name a few). If you attempt to install both stacks on the same cluster of servers, you will experience conflicts.

[#7]

@Mourad Chahri

Different Ambari servers can not own the same hosts/nodes. Ambari agents which are installed on each node are configured to communicate with a single Ambari Server.

[#8]

@Mourad Chahri

"how to install HDF on the same cluster , because i wanna use HDF and HDP"

HDF does not need to be installed on the same hardware as HDP in order to have the software packages send data to one another.

For example, HDF NiFi includes the hadoop client libraries needed to send/get data from HDP HDFS. All you need to provide NiFi is the core-sites.xml and HDFS-sites.xml files. No need to install Hadoop (HDFS) clients on the NiFi nodes/hosts or have HDP HDFS installed on the same nodes/hosts.

Thanks,

Matt

[#9]

@Matt Clarke because i will need hbase also so i dont think that i can add it in hdp

[#10]

@Mourad Chahri

Hbase runs on top of HDFS in HDP. The only service that is part of HDF that is not in HDP is NiFi. NiFi can send and retrieve data with and HDP HDFS with Hbase without both services needing to be installed on the same nodes/hosts.