Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

It's sometimes useful to adjust the log level when executing commands from the client to get more info about what is happening. This article explains how that can be achieved:

You can set DEBUG level for any client command by exporting the following environment variable before it is executed:

HADOOP_ROOT_LOGGER=hadoop.root.logger=DEBUG,console

Now try executing a client command and watch the stream of DEBUG info come to your terminal. For example:

# hdfs dfs -ls

16/12/24 15:01:34 DEBUG util.Shell: setsid exited with exit code 0
16/12/24 15:01:34 DEBUG conf.Configuration: parsing URL jar:file:/usr/hdp/2.5.3.0-37/hadoop/hadoop-common-2.7.3.2.5.3.0-37.jar!/core-default.xml
16/12/24 15:01:34 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@149494d8
16/12/24 15:01:34 DEBUG conf.Configuration: parsing URL file:/etc/hadoop/2.5.3.0-37/0/core-site.xml
16/12/24 15:01:34 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@28c4711c
16/12/24 15:01:35 DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use_ip to true
16/12/24 15:01:35 DEBUG security.Groups:  Creating new Groups object
16/12/24 15:01:35 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
16/12/24 15:01:35 DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
16/12/24 15:01:35 DEBUG security.JniBasedUnixGroupsMapping: Using JniBasedUnixGroupsMapping for Group resolution
16/12/24 15:01:35 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMapping
16/12/24 15:01:35 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
16/12/24 15:01:35 DEBUG security.UserGroupInformation: hadoop login
16/12/24 15:01:35 DEBUG security.UserGroupInformation: hadoop login commit
16/12/24 15:01:35 DEBUG security.UserGroupInformation: using kerberos user:hdfs@HWX.COM
16/12/24 15:01:35 DEBUG security.UserGroupInformation: Using user: "hdfs@HWX.COM" with name hdfs@HWX.COM
16/12/24 15:01:35 DEBUG security.UserGroupInformation: User entry: "hdfs@HWX.COM"
16/12/24 15:01:35 DEBUG security.UserGroupInformation: UGI loginUser:hdfs@HWX.COM (auth:KERBEROS)
<snip> 

Note, the following log levels are supported: ALL DEBUG ERROR FATAL INFO OFF TRACE TRACE_INT WARN

5,141 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
3 of 3
Last update:
‎02-24-2020 02:09 AM
Updated by:
 
Contributors
Top Kudoed Authors