How to copy files from HDFS to remote server

[#1]

Hi,

I am trying to copy files from HDFS location to a folder in the remote server.

I have tried using hdfs dfs -get/-copyToLocal <hdfs path> <local path> commands, yet no luck.

I need to make this script run automatically everyday. How do i do that?

I am very new to Hadoop. Any help is greatly appreciated

[#2]

Can you not install hdfs client (just binaries) on the remote server and run a cron job to get hdfs data you want locally? That way you don't need to cache data locally and then rsync/scp to remote server?

SDL

[#3]

Hi @Naush007 When you say remote server you mean a different host in the same network but on different cluster ? -copyToLocal is just for the same host on which you have the data. It will not copy to a different host. To do this you can use "DistCp" Distributed Copy tool to copy the data you want. Refer to below links for more information and how to use them.

https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html

https://docs.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrat...

- If this is not what you are looking for please let us know what exactly needs to be achieved here. Thank you. If this helps you don't forget to click on accepted solution.