- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
distcp not distributing
- Labels:
-
HDFS
Created on 05-01-2015 09:24 AM - edited 09-16-2022 02:27 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm transferring files using distcp on Cloudera 5.3.x, and I can't get it to distribute the transfer using MR2.
I don't have MR1 installed at all. I'd rather not because it'll hide issues.
My command line looks like this, and it runs just fine, it just copies every file in series:
mapred distcp s3n://key:secret@logs.space.com/source/2015/04/28/ hdfs://nameservice/target/dir/2015/04/28
Is there a configuration item that I missed?
Created 05-01-2015 10:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
valid RM configs under /etc/hadoop/conf/yarn-site.xml?
Do you see the word 'LocalJobRunner' in the output logs of the DistCp
command when its running?
Created 05-01-2015 10:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
valid RM configs under /etc/hadoop/conf/yarn-site.xml?
Do you see the word 'LocalJobRunner' in the output logs of the DistCp
command when its running?
Created 05-01-2015 02:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That was it! Thank you!
