Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to delete the /var/log partition permanently?

Highlighted

How to delete the /var/log partition permanently?

New Contributor

The node manager in the primary node was down and when I tried to start it I got the below error.

==> /var/log/hadoop-yarn/yarn/yarn-yarn-nodemanager-hbase.primary.namenode.out.1 <==
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:176)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger.log(NMLeveldbStateStoreService.java:1032)
at org.fusesource.leveldbjni.JniDBFactory$OptionsResourceHolder$2.log(JniDBFactory.java:143) 

When I checked the disk space I noticed some new partitions are mounted after the cluster setup. "/var/log" is mounted as a new partition with only 976MB and it is 100% used. I was able to start node manager after deleting old log files. But the error repeats when the disk is full.

How can I get rid of this problem?

Why does ambari create a new partitions?

Can we delete "/var/log" partition permanently because the "/var" partition has enough disk space so I don't need to change the log paths?

3 REPLIES 3

Re: How to delete the /var/log partition permanently?

Mentor

@Thanuja Kularathna

Have a look at this example

Re: How to delete the /var/log partition permanently?

Super Collaborator

Hi @Thanuja Kularathna,

you may do two things

1. Update Ambari Configuration(log4j) to point to new location where you have more space(do it for all the services). - proper way

2. just remove the partition and create a sim link to respective location where you have plenty of Space - hack around

either the cases you can get rid of the problem.

Re: How to delete the /var/log partition permanently?

New Contributor

I updated the log paths of each service in log4j settings. Seems to be a safer and a simpler way than playing around with the partitions or symbolic links.

Don't have an account?
Coming from Hortonworks? Activate your account here