Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to copy files from local desktop lo hdfs location using NiFi?

avatar
 
1 ACCEPTED SOLUTION

avatar
Super Mentor
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

View solution in original post

2 REPLIES 2

avatar
Super Mentor
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

avatar
Master Guru

@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi.