question Re: How to configure ListHDFS Nifi on a machine to list all file from HDFS server on other machine? in Archives of Support Questions (Read Only)

How to configure ListHDFS Nifi on a machine to list all file from HDFS server on other machine?

kiemnguyenxuan1 — Fri, 02 Jun 2017 10:07:01 GMT

I have a Nifi cluster on machine A, B, C and a HDFS standalone on machine D. Now I want to get all file from HDFS server and push into nifi cluster to execute jobs.

Please help me to configure this dataflow and how to setup properties on nifi processor.

Thanks,

Re: How to configure ListHDFS Nifi on a machine to list all file from HDFS server on other machine?

MattWho — Sat, 03 Jun 2017 01:13:00 GMT

@Kiem Nguyen

The documentation for the ListHDFS processor can be found imbedded in the imbedded documetation that is accessible via "help".

Latest Apache version doc can also be found here:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.2.0/org.apache.nifi.processors.hadoop.ListHDFS/index.html

You need to get a copy of your core-site.xml and hdfs-site.xml files from your HDFS install and put them in the same place on every NiFi node to be referenced by this processor.

Thanks,

Matt

Re: How to configure ListHDFS Nifi on a machine to list all file from HDFS server on other machine?

kiemnguyenxuan1 — Mon, 05 Jun 2017 11:08:14 GMT

It works well. Thanks Matt !!!

Hadoop Configuration Resources Properties: point to core-site.xml and hdfs-site.xml after copying them into each Nifi node.