how to copy file from remote server to HDFS

[#1]

I have a remote server and servers authenticated Hadoop environment.

I want to copy file from Remote server to Hadoop machine to HDFS

Please advise efficient approach/HDFS command to copy files from remote server to HDFS.

Any example will be helpful.

as ordinary way to copy file from remote server to server itself is

scp -rp file remote_server:/tmp

but this approach not support copy directly to hdfs

[#2 · Accepted solution]

@mike_bronson7 Are you trying to copy local files from your remote machine to destination HDFS cluster? You could use distcp if it is between hdfs. Please refer below documentation.

https://docs.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrat...

For the local files in remote machine, you could SCP the files to any of the cluster node which has hdfs client installed and the do a "-copyFromLocal" or "-put" to push that to HDFS. Hope this helps.

[#3]

Can you please give an example for this?

For the local files in remote machine, you could SCP the files to any of the cluster node which has hdfs client installed and the do a "-copyFromLocal" or "-put" to push that to HDFS. Hope this helps.

Thanks

[#4]

I was looking for the same info and found that great link below.

https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html

I hope it can help you.

Best,