Best Way to Persist NameNode Log Level?

KCJeffro — Thu, 25 Aug 2022 15:22:27 GMT

On CDP Private Cloud 7.1.7, and running without rack awareness (an intentional choice due to some of the older servers running with smaller data disks), I am (as would be expected) seeing this spammed to the namenode logs:

2022-08-24 09:41:50,253 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not enough replicas was chosen. Reason:{NO_REQUIRED_STORAGE_TYPE=1}

Because I'm not logging INFO messages, I can eliminate those entries with something like:

hadoop daemonlog -setlevel nnserverhost:9870 org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy WARN

What's the best way to persist that across a restart?

Re: Best Way to Persist NameNode Log Level?

rki_ — Thu, 25 Aug 2022 15:44:56 GMT

Hi @KCJeffro ,

The best possible way would be to change the log level for the thread 'org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy' to 'ERROR'. Do the following:

Navigate to Cloudera Manager > HDFS > Config > search for 'NameNode Logging Advanced Configuration Snippet (Safety Valve) log4j_safety_valve'

Add the following property:

log4j.logger.org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy=ERROR

question Re: Best Way to Persist NameNode Log Level? in Support Questions

Best Way to Persist NameNode Log Level?

Re: Best Way to Persist NameNode Log Level?