Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

uber mode not activated on CDH5

uber mode not activated on CDH5

New Contributor

When launching a  map reduce job for processing 2000 smal pdf files (0.7 Mb) the uver mode does not get activated. See more information below:

 

14/05/04 15:53:00 INFO input.FileInputFormat: Total input paths to process : 2000
14/05/04 15:53:00 INFO input.CombineFileInputFormat: DEBUG: Terminated node allocation with : CompletedNodes: 1, size left: 27516040
14/05/04 15:53:01 INFO mapreduce.JobSubmitter: number of splits:21
14/05/04 15:53:02 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1399142719676_0002
14/05/04 15:53:03 INFO impl.YarnClientImpl: Submitted application application_1399142719676_0002
14/05/04 15:53:03 INFO mapreduce.Job: The url to track the job: http://localhost.localdomain:8088/proxy/application_1399142719676_0002/
14/05/04 15:53:03 INFO mapreduce.Job: Running job: job_1399142719676_0002
14/05/04 15:54:42 INFO mapreduce.Job: Job job_1399142719676_0002 running in uber mode : false

 

yarn-conf/mapred-site.xml

 

<property><name>mapreduce.job.ubertask.enabled</name><value>true</value></property><property><name>mapreduce.job.ubertask.maxmaps</name><value>100</value></property><property><name>mapreduce.job.ubertask.maxreduces</name><value>0</value></property><property><name>mapreduce.job.ubertask.maxbytes</name><value>167108864</value></property>

 

Any ideas?

 

Many thanks.

1 REPLY 1

Re: uber mode not activated on CDH5

Master Guru
I assume by 0.7 MB you meant that 2000 files totally consume 0.7 MB, and not per file being 0.7 MB?

You may need to additionally ensure the below rules via your configuration:

Memory of AM via (yarn.app.mapreduce.am.resource.mb) must be greater than values of task memory via (mapreduce.map.memory.mb and mapreduce.reduce.memory.mb)

CPU vCores request of AM via (yarn.app.mapreduce.am.resource.cpu-vcores) must be greater than the values of task vCores request via (mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores)