question Re: Install apache NIFI with ambari on existing HDP cluster in Support Questions

Install apache NIFI with ambari on existing HDP cluster

chahrimourad — Fri, 24 Feb 2017 20:08:31 GMT

Please how to install apache NIFI with ambari on existing HDP cluster

Re: Install apache NIFI with ambari on existing HDP cluster

myoung — Fri, 24 Feb 2017 20:11:46 GMT

If you are looking to demo just NiFi on an HDP cluster, then you can use this: https://github.com/abajwa-hw/ambari-nifi-service

It is unsupported and not intended for production uses. If you want to deploy Hortonworks Data Flow which includes NiFi, then you currently have to install it using a separate version of Ambari from the HDP Ambari.

Re: Install apache NIFI with ambari on existing HDP cluster

chahrimourad — Fri, 24 Feb 2017 20:18:24 GMT

@Michael Young how to install HDF on the same cluster , because i wanna use HDF and HDP

Re: Install apache NIFI with ambari on existing HDP cluster

myoung — Fri, 24 Feb 2017 20:30:14 GMT

@Mourad Chahri

You can not currently install HDP and HDF on the same cluster because there are conflicts with Ambari and the service configurations. You must install HDF and HDP on different servers.

Re: Install apache NIFI with ambari on existing HDP cluster

chahrimourad — Fri, 24 Feb 2017 20:35:01 GMT

@Michael Young can i use the same cluster but , install ambari server fo hdp on a server and ambari-server for HDF on another server but use the same node ??

Re: Install apache NIFI with ambari on existing HDP cluster

myoung — Fri, 24 Feb 2017 20:43:17 GMT

@Mourad Chahri

That is not currently a supported configuration. There are common components in HDF and HDP (Kafka, Storm, Ranger, to name a few). If you attempt to install both stacks on the same cluster of servers, you will experience conflicts.

Re: Install apache NIFI with ambari on existing HDP cluster

MattWho — Fri, 24 Feb 2017 20:47:45 GMT

@Mourad Chahri

Different Ambari servers can not own the same hosts/nodes. Ambari agents which are installed on each node are configured to communicate with a single Ambari Server.

Re: Install apache NIFI with ambari on existing HDP cluster

MattWho — Fri, 24 Feb 2017 20:52:07 GMT

@Mourad Chahri

"how to install HDF on the same cluster , because i wanna use HDF and HDP"

HDF does not need to be installed on the same hardware as HDP in order to have the software packages send data to one another.

For example, HDF NiFi includes the hadoop client libraries needed to send/get data from HDP HDFS. All you need to provide NiFi is the core-sites.xml and HDFS-sites.xml files. No need to install Hadoop (HDFS) clients on the NiFi nodes/hosts or have HDP HDFS installed on the same nodes/hosts.

Thanks,

Matt

Re: Install apache NIFI with ambari on existing HDP cluster

chahrimourad — Fri, 24 Feb 2017 21:51:00 GMT

@Matt Clarke because i will need hbase also so i dont think that i can add it in hdp

Re: Install apache NIFI with ambari on existing HDP cluster

MattWho — Fri, 24 Feb 2017 22:30:11 GMT

@Mourad Chahri

Hbase runs on top of HDFS in HDP. The only service that is part of HDF that is not in HDP is NiFi. NiFi can send and retrieve data with and HDP HDFS with Hbase without both services needing to be installed on the same nodes/hosts.

Re: Install apache NIFI with ambari on existing HDP cluster

chahrimourad — Fri, 24 Feb 2017 22:36:39 GMT

@Matt Clarke so what you proposition to use hortonworks with IOT ?

have two cluster one for HDP and the second for HDF or use just HDF and other component

Re: Install apache NIFI with ambari on existing HDP cluster

marcus_aidley — Tue, 26 Sep 2017 16:27:14 GMT

As of HDP 2.6.1 it is possible to install HDF components on an HDP cluster. See https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_installing-hdf-and-hdp/content/ch_install-ambari.html