- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How do I set log level of namenode to DEBUG?
- Labels:
-
Apache Hadoop
Created on ‎02-07-2019 08:33 PM - edited ‎08-17-2019 02:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the picture I attached to this post you can see my current log level value. This does not work. In /var/log/hadoop/hdfs/hadoop-hdfs-namenode-sandbox-hdp.hortonworks.com.log I can only see INFO and WARN messages.
Created ‎02-07-2019 11:02 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can change the log level while logged on as hdfs with the below steps, you don't need to restart the namenode
Get the current log level
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO
Change to DEBUG
$ hadoop daemonlog -setlevel {namenode_host}:50070 BlockStateChange DEBUG
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=DEBUG Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Submitted Level: DEBUG Setting Level to DEBUG ... Effective Level: DEBUG
Validate DEBUG mode
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: DEBUG
You should be able to notice the logging level in namenode.log has been updated, without restarting the service. After finishing your diagnostics you can reset the logging level back to INFO
Reset to INFO
$ hadoop daemonlog -setlevel {namenode_host}:50070 BlockStateChange INFO
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=INFO Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Submitted Level: INFO Setting Level to INFO ... Effective Level: INFO
Validate INFO
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO
There you go
If you found this answer addressed your question, please take a moment to log
in and click the "accept" link on the answer.
Created ‎02-07-2019 11:02 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can change the log level while logged on as hdfs with the below steps, you don't need to restart the namenode
Get the current log level
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO
Change to DEBUG
$ hadoop daemonlog -setlevel {namenode_host}:50070 BlockStateChange DEBUG
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=DEBUG Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Submitted Level: DEBUG Setting Level to DEBUG ... Effective Level: DEBUG
Validate DEBUG mode
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: DEBUG
You should be able to notice the logging level in namenode.log has been updated, without restarting the service. After finishing your diagnostics you can reset the logging level back to INFO
Reset to INFO
$ hadoop daemonlog -setlevel {namenode_host}:50070 BlockStateChange INFO
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=INFO Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Submitted Level: INFO Setting Level to INFO ... Effective Level: INFO
Validate INFO
$ hadoop daemonlog -getlevel {namenode_host}:50070 BlockStateChange
Output
Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange Submitted Log Name: BlockStateChange Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO
There you go
If you found this answer addressed your question, please take a moment to log
in and click the "accept" link on the answer.
Created ‎02-08-2019 11:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey, thanks so much!
Created ‎09-30-2020 11:50 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you please help here. When I give "-protocol https", getting below error
Connecting to https://HOST:PORT/logLevel?log=BlockStateChange
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://HOST:PORT/logLevel?log=BlockStateChange
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
at org.apache.hadoop.log.LogLevel$CLI.process(LogLevel.java:297)
at org.apache.hadoop.log.LogLevel$CLI.doGetLevel(LogLevel.java:234)
at org.apache.hadoop.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:127)
at org.apache.hadoop.log.LogLevel$CLI.run(LogLevel.java:110)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.log.LogLevel.main(LogLevel.java:72)
