Member since
10-30-2018
3
Posts
0
Kudos Received
0
Solutions
10-04-2019
05:01 AM
It works only if it's the same KDC, need cross trust (realm) in my case. Thank you. ps : I didn't got the notification too Regards
... View more
10-30-2018
05:38 PM
Hi, You can find here all the options for distcp : https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html To improve that, i used -strategy dynamic and increased the number of the mappers (-m ) also the bandwith per mapper (-bandwith) ans the size of your containers of course if you want customize it. so you finally have : hadoop distcp -Dmapreduce.map.memory.mb=4096-Dyarn.app.mapreduce.am.resource.mb=4096 -Dmapred.job.queue.name=DISTCP_exec -prb -bandwidth 50 -m 16 -update -delete -strategy dynamic hdfs://source/path/.snapshot/20181030-170124.063 swebhdfs://target/path
... View more