Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

how to copy files from local desktop lo hdfs location using NiFi?

 
1 ACCEPTED SOLUTION

Master Guru
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

View solution in original post

2 REPLIES 2

Master Guru
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

Super Guru

@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi.