https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229657#M82525 <A rel="user" href="https://community.cloudera.com/users/56174/saikrishnamakineni-1.html" nodeid="56174">@Sai Krishna Makineni</A><P>You can use <STRONG>List/Fetch SFTP processors</STRONG> and then Store the Fetched files into HDFS by using PutHDFS processor.</P><P><STRONG><U>Flow:</U></STRONG></P><PRE>1.ListSFTP (or) ListFTP //list the files fro remote path<BR />2.Remote Processor group //distribute the load across the cluster<BR />3.FetchSFTP (or) FetchFTP //fetch the files from remote<BR />4.PutHDFS //store the fetched files into HDFS</PRE><P>Please refer to <A href="https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi/" target="_blank">this</A> link for more details regards to fetching files from remote path and usage of List/Fetch processors in NiFi.</P> Wed, 22 Aug 2018 05:23:56 GMT Shu_ashu 2018-08-22T05:23:56Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229656#M82524 <P>I would like to fetch files from remote server from ../abc/xyz/kpi path and put it to HDFS and the files in the remote path is getting updated everyday. Being a new person to NiFi I would like to understand which are best processors that can help me here.</P> Wed, 22 Aug 2018 04:45:20 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229656#M82524 saikrishnamakin 2018-08-22T04:45:20Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229657#M82525 <A rel="user" href="https://community.cloudera.com/users/56174/saikrishnamakineni-1.html" nodeid="56174">@Sai Krishna Makineni</A><P>You can use <STRONG>List/Fetch SFTP processors</STRONG> and then Store the Fetched files into HDFS by using PutHDFS processor.</P><P><STRONG><U>Flow:</U></STRONG></P><PRE>1.ListSFTP (or) ListFTP //list the files fro remote path<BR />2.Remote Processor group //distribute the load across the cluster<BR />3.FetchSFTP (or) FetchFTP //fetch the files from remote<BR />4.PutHDFS //store the fetched files into HDFS</PRE><P>Please refer to <A href="https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi/" target="_blank">this</A> link for more details regards to fetching files from remote path and usage of List/Fetch processors in NiFi.</P> Wed, 22 Aug 2018 05:23:56 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229657#M82525 Shu_ashu 2018-08-22T05:23:56Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229658#M82526 <P><A href="https://www.welookups.com">https://www.welookups.com</A></P> Thu, 23 Aug 2018 23:39:29 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Fetch-files-from-remote-server-and-put-it-to-HDFS/m-p/229658#M82526 Former Member 2018-08-23T23:39:29Z