Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Node Manager Log Directory Free Space

Highlighted

Node Manager Log Directory Free Space

New Contributor

Hi there

I am new to Cloudera Hadoop.

I need to know how can I free space from Container Log Directories in Node Manager.

3 REPLIES 3

Re: Node Manager Log Directory Free Space

Master Collaborator
Delete the old unused log directories. The directories are in the form of application_<timestamp>_sequence
So you can easily write a script to remove everything older than X days

Re: Node Manager Log Directory Free Space

New Contributor
I tried to move logs from /var/log/hadoop-yarn but no use. Can you tell me the path of the unused log directories.
Thanks

Re: Node Manager Log Directory Free Space

Expert Contributor

To review the state please

 

  • Get the affected directory path from CM > Configuration > Scope: YARN (MR2 included) > NodeManager Log Directory. The default is /var/log/hadoop-yarn
  • Verify the available disk space for this path:
    # df -h /var/log/hadoop-yarn
  • Look up what are the biggest disk space consumers on this mount point
  • Clean up where possible

 

If no clean up is possible and there is sufficient disk space available then you can in decrease the alarm threshold in the CM > YARN > Configuration > Scope: Node Manager > Category: Monitoring > Log Directory Free Space Monitoring Percentage Thresholds configuration property.