Reply
New Contributor
Posts: 2
Registered: ‎03-28-2019

Node Manager Log Directory Free Space

Hi there

I am new to Cloudera Hadoop.

I need to know how can I free space from Container Log Directories in Node Manager.

Master
Posts: 430
Registered: ‎07-01-2015

Re: Node Manager Log Directory Free Space

Delete the old unused log directories. The directories are in the form of application_<timestamp>_sequence
So you can easily write a script to remove everything older than X days
New Contributor
Posts: 2
Registered: ‎03-28-2019

Re: Node Manager Log Directory Free Space

I tried to move logs from /var/log/hadoop-yarn but no use. Can you tell me the path of the unused log directories.
Thanks
Highlighted
Cloudera Employee
Posts: 222
Registered: ‎01-15-2015

Re: Node Manager Log Directory Free Space

To review the state please

 

  • Get the affected directory path from CM > Configuration > Scope: YARN (MR2 included) > NodeManager Log Directory. The default is /var/log/hadoop-yarn
  • Verify the available disk space for this path:
    # df -h /var/log/hadoop-yarn
  • Look up what are the biggest disk space consumers on this mount point
  • Clean up where possible

 

If no clean up is possible and there is sufficient disk space available then you can in decrease the alarm threshold in the CM > YARN > Configuration > Scope: Node Manager > Category: Monitoring > Log Directory Free Space Monitoring Percentage Thresholds configuration property.

Announcements