Created 01-09-2018 05:01 PM
Hello All,
I have a requirement where i want to copy files from one hdfs directory to another via oozie in same cluster.
This can be done using oozie discp action or oozie shell action.
Which is a better way to copy files using oozie.
I guess it is similar as asking hdfs -cp vs distcp?
Thanks and Best Regards,
Gagan
Created 01-09-2018 05:22 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 01-09-2018 05:22 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 01-10-2018 08:35 AM
This is very much the same i researched too. So i go with distcp for my usecase.
Created 01-10-2018 09:19 AM
All,
Just adding for knowledge gain , if my source is kerberos enabled while target is not , then the command to be executed will be
hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true webhdfs://source-ip webhdfs://target-ip
hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true : this command overrides values present in the hive-site.xml.
thanks,
Rishit Shah