Support Questions
Find answers, ask questions, and share your expertise

I am using Hortonworks distribution HDP 2.6.4,HDF,How to Interact with HDFS data platform by using NIFI.

Highlighted

I am using Hortonworks distribution HDP 2.6.4,HDF,How to Interact with HDFS data platform by using NIFI.

I have Data Platform HDFS in HDP,NIFI in HDF

Now to interact HDFS by using NIFI(Like Put HDFS),

1.Is nifi should be added service on HDP platform

or

2.Can we achieve interaction even they are in different (HDP and HDF),if yes how to proceed

Kindly help me out as i am new to nifi.

3 REPLIES 3
Highlighted

Re: I am using Hortonworks distribution HDP 2.6.4,HDF,How to Interact with HDFS data platform by using NIFI.

Super Mentor

@Preethi Sanduru
Option-1). Copy and paste the "hdfs-site.xml" and "core-site.xml" files from your HDP installation to HDF insiallation's "/usr/hdf/current/nifi/lib" path and restart nifi

Option-2). You can copy "hdfs-site.xml" and "core-site.xml" files from HDP cluster and then put it to some opath on the HDF cluster host where the NIFI is installed. Then specify the path in the HDFS processor configurations then nifi will refer to those configuration resources from your specified path. "Hadoop Configuration Resources property" of your PutHDFS processor you can pass the path of those XML files.

Hadoop Configuration Resources property = /etc/hdp/hdfs-site.xml,/etc/hdp/core-site.xml

Reference:

https://community.hortonworks.com/questions/141182/standard-way-of-giving-hadoop-configuration-resou...

.

Highlighted

Re: I am using Hortonworks distribution HDP 2.6.4,HDF,How to Interact with HDFS data platform by using NIFI.

Thanks for quick response @Jay Kumar SenSharma

Is there any particular file format i need to use while copying and pasting configuration files into HDF path from HDP.

Highlighted

Re: I am using Hortonworks distribution HDP 2.6.4,HDF,How to Interact with HDFS data platform by using NIFI.

Super Mentor

@Preethi Sanduru

The configuration files are in XML format. So we can simply "scp" the files from HDP cluster to the HDF cluster Nifi host.