Skip to main content
KCJeffro
KCJeffro
Bronze ⭐️⭐️⭐️
August 25, 2022
Solved

Best Way to Persist NameNode Log Level?

On CDP Private Cloud 7.1.7, and running without rack awareness (an intentional choice due to some of the older servers running with smaller data disks), I am (as would be expected) seeing this spammed to the namenode logs:
 
2022-08-24 09:41:50,253 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not enough replicas was chosen. Reason:{NO_REQUIRED_STORAGE_TYPE=1}
 
Because I'm not logging INFO messages, I can eliminate those entries with something like:
 
hadoop daemonlog -setlevel nnserverhost:9870 org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy WARN
 
What's the best way to persist that across a restart?
rki_
rki_Answer
Gold ⭐️
August 25, 2022

Hi @KCJeffro ,

 

The best possible way would be to change the log level for the thread 'org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy' to 'ERROR'. Do the following:

  1. Navigate to Cloudera Manager > HDFS > Config > search for 'NameNode Logging Advanced Configuration Snippet (Safety Valve) log4j_safety_valve'
  2. Add the following property:

    log4j.logger.org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy=ERROR

View in thread

1 reply

rki_
Gold ⭐️
rki_Answer
Gold ⭐️
August 25, 2022

Hi @KCJeffro ,

 

The best possible way would be to change the log level for the thread 'org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy' to 'ERROR'. Do the following:

  1. Navigate to Cloudera Manager > HDFS > Config > search for 'NameNode Logging Advanced Configuration Snippet (Safety Valve) log4j_safety_valve'
  2. Add the following property:

    log4j.logger.org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy=ERROR

Translate
Terms & ConditionsAccessibility statement