Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to change the log level of a MR job

Solved Go to solution
Highlighted

How to change the log level of a MR job

New Contributor

Hi,

I am trying to change the log level of a MR job, I used the following command : 

hadoop jar appMR.jar MainMR -Dmapreduce.map.log.level=DEBUG <args...>

but the -Dmapreduce.map.log.level=DEBUG goes as the first argument to the job.  

 Is there any way to do this only for specific MR jobs without changes the level in the log4j properties file ? 
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to change the log level of a MR job

Master Guru
You may only use the -Dname=value form if your main class implements the
Tool interface and gets invoked via the ToolRunner utility. Check the Tool
javadoc example and model your implementation around it:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/util/Tool.html

View solution in original post

2 REPLIES 2
Highlighted

Re: How to change the log level of a MR job

Super Collaborator

To turn on debug mode on the job level, issue the following command before executing the job:
export HADOOP_ROOT_LOGGER=hadoop.root.logger=Debug,console
or add -Dhadoop.root.logger=DEBUG,console"

 

Highlighted

Re: How to change the log level of a MR job

Master Guru
You may only use the -Dname=value form if your main class implements the
Tool interface and gets invoked via the ToolRunner utility. Check the Tool
javadoc example and model your implementation around it:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/util/Tool.html

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here