Member since
02-27-2025
3
Posts
0
Kudos Received
0
Solutions
03-11-2025
10:23 AM
@VidyaSargur i dont have an access to the article waiting for share solution.
... View more
03-11-2025
10:23 AM
@willx i really appreciate for response, looks like i don't have an access to the Article. https://my.cloudera.com/knowledge/How-to-collect-thread-dumps-for-stuck-YARN-containers-via-jstack?i... Can you please share the solution i really appreciate for help. Thanks, Zubair.
... View more
03-02-2025
11:51 PM
Hi, Currently i have 1 TB of data in HDFS where i am trying to migrate into S3, i am using below command, when ever i run this command job runs very fast for 3 hours then it slows down for a week still it is running, i started last week to run this job still it is running and very slow, is this expected behavior. nohup hadoop distcp -Dfs.s3a.access.key="$AWS_ACCESS_KEY_ID" -Dfs.s3a.secret.key="$AWS_SECRET_ACCESS_KEY" -Dfs.s3a.fast.upload=true -Dfs.s3a.fast.buffer.size=1048576 -Dfs.s3a.multipart.size=10485760 -Dfs.s3a.multipart.threshold=10485760 -Dmapreduce.map.memory.mb=8192 -Dmapreduce.map.java.opts=-Xmx7360m -m=300 -bandwidth 400 -update hdfs:<....> s3a://<.......>
... View more
Labels:
- Labels:
-
HDFS