question Re: How to change the log level of a MR job in Archives of Support Questions (Read Only)

How to change the log level of a MR job

tsialiam — Fri, 25 Aug 2017 18:02:23 GMT

Hi,

I am trying to change the log level of a MR job, I used the following command :

hadoop jar appMR.jar MainMR -Dmapreduce.map.log.level=DEBUG <args...>

but the -Dmapreduce.map.log.level=DEBUG goes as the first argument to the job.

Is there any way to do this only for specific MR jobs without changes the level in the log4j properties file ?

Re: How to change the log level of a MR job

Fawze — Sun, 27 Aug 2017 04:25:40 GMT

To turn on debug mode on the job level, issue the following command before executing the job:
export HADOOP_ROOT_LOGGER=hadoop.root.logger=Debug,console
or add -Dhadoop.root.logger=DEBUG,console"

Re: How to change the log level of a MR job

Harsh J — Sun, 27 Aug 2017 04:46:55 GMT

You may only use the -Dname=value form if your main class implements the
Tool interface and gets invoked via the ToolRunner utility. Check the Tool
javadoc example and model your implementation around it:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/util/Tool.html