Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎07-13-2017
Accepted Solution

How to change the log level of a MR job

Hi,

I am trying to change the log level of a MR job, I used the following command : 

hadoop jar appMR.jar MainMR -Dmapreduce.map.log.level=DEBUG <args...>

but the -Dmapreduce.map.log.level=DEBUG goes as the first argument to the job.  

 Is there any way to do this only for specific MR jobs without changes the level in the log4j properties file ? 
Expert Contributor
Posts: 253
Registered: ‎01-25-2017

Re: How to change the log level of a MR job

To turn on debug mode on the job level, issue the following command before executing the job:
export HADOOP_ROOT_LOGGER=hadoop.root.logger=Debug,console
or add -Dhadoop.root.logger=DEBUG,console"

 

Posts: 1,524
Kudos: 265
Solutions: 232
Registered: ‎07-31-2013

Re: How to change the log level of a MR job

You may only use the -Dname=value form if your main class implements the
Tool interface and gets invoked via the ToolRunner utility. Check the Tool
javadoc example and model your implementation around it:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/util/Tool.html
Backline Customer Operations Engineer
Announcements