Support Questions

Find answers, ask questions, and share your expertise

how to copy file from remote server to HDFS

 
 

I have a remote server and servers authenticated Hadoop environment.

I want to copy file from Remote server to Hadoop machine to HDFS

Please advise efficient approach/HDFS command to copy files from remote server to HDFS.

Any example will be helpful.

as ordinary way to copy file from remote server to server itself is

scp -rp file remote_server:/tmp

but this approach not support copy directly to hdfs

Michael-Bronson
1 ACCEPTED SOLUTION

Cloudera Employee

@mike_bronson7 Are you trying to copy local files from your remote machine to destination HDFS cluster? You could use distcp if it is between hdfs. Please refer below documentation.

https://docs.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrat...

 

For the local files in remote machine, you could SCP the files to any of the cluster node which has hdfs client installed and the do a "-copyFromLocal" or "-put" to push that to HDFS. Hope this helps.

View solution in original post

2 REPLIES 2

Cloudera Employee

@mike_bronson7 Are you trying to copy local files from your remote machine to destination HDFS cluster? You could use distcp if it is between hdfs. Please refer below documentation.

https://docs.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrat...

 

For the local files in remote machine, you could SCP the files to any of the cluster node which has hdfs client installed and the do a "-copyFromLocal" or "-put" to push that to HDFS. Hope this helps.

New Contributor

Can you please give an example for this?

 

For the local files in remote machine, you could SCP the files to any of the cluster node which has hdfs client installed and the do a "-copyFromLocal" or "-put" to push that to HDFS. Hope this helps.

 

Thanks

 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.