Can we use sqoop to Transfer file from a HDFS directory to a remote server.

[#1]

We have a requirement to transfer files from an HDFS directory to a remote server.

I've noticed options to copy files from HDFS to the local filesystem first (using copyToLocal) and then transfer files from the local filesystem to the remote server (using scp). But is there any direct method to copy files from HDFS to a remote server, such as using Sqoop functionality or any other methods like without copying to the local file system

[#2]

Hi @s198 

You can use DistCp command to achieve the same.

refer ---> https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html

[#3]

Thanks @jAnshula for your suggestion

My remote server does not support Hadoop-compatible file systems, so the DistCp command does not work for me. The primary objective is to copy the HDFS files as is to a Linux machine. 

[#4]

@s198 

You may try and use NIFI, refer below article

https://community.cloudera.com/t5/Support-Questions/How-to-copy-files-from-a-remote-linux-box-using-...

[#5]

Thanks @jAnshula for your suggestion.
We are trying to achieve this without using Nifi. Could you please let us know if any options are available using hdfs/hadoop commands?

[#6]

Hi @s198, You do not need to have hadoop file system or datanode role on the remote server. You just need to set up some hdfs gateway on the remote server and pull it using distcp. If you are using HDP or CDP, you can add the remote server as a gateway and perform distcp in the remote server.

Another option is to share one of the directories in the remote server, mount it in hadoop cluster node, and perform distcp to that mounted directory.