Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to copy files from local desktop lo hdfs location using NiFi?

Solved Go to solution
Highlighted

how to copy files from local desktop lo hdfs location using NiFi?

 
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: how to copy files from local desktop lo hdfs location using NiFi?

Master Guru
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

View solution in original post

2 REPLIES 2
Highlighted

Re: how to copy files from local desktop lo hdfs location using NiFi?

Master Guru
@kishore sanchina

The simplest answer to your question is to use the ListFile processor to produce a list of the files from your local filesystem, feed that to a fetchFile processor that will pickup the content and then pass them to a PutHDFS processor to send them to your HDFS. The listFile processor will maintain state based on lastModified time on the files to ensure the files are not listed more then once.

6969-screen-shot-2016-08-25-at-105659-am.png

If you right click on either of these NiFi processors you can select "usage" from the displayed context menu to get more details on the configuration of each of these.

Thanks,

Matt

View solution in original post

Re: how to copy files from local desktop lo hdfs location using NiFi?

Super Guru

@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi.

Don't have an account?
Coming from Hortonworks? Activate your account here