How to Copy Files from Local Directory to HDFS Using Apache NiFi (ListFile, FetchFile, PutHDFS)

[#1]

how to copy files from local desktop lo hdfs location using NiFi?

[#2 · Accepted solution]

@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

[#3]

@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi.