space issue in /var/lib/cloudera*

[#1]

we are facing space issue on following directories. it is safe to remove or move these directories to another directory or make a tar of it. i need space . PLease help

[root@mprl509 lib]#pwd
/var/lib/
[root@mprl509 lib]# du -sh cloudera-*
1.3G cloudera-host-monitor
300K cloudera-scm-agent
20M cloudera-scm-eventserver
4.6M cloudera-scm-headlamp
3.0G cloudera-scm-server
272M cloudera-scm-server-db
5.5G cloudera-service-monitor

[#2]

@itsmezeeshan, Welcome to our community! To help you get the best possible answer, I have tagged in our CDP experts @rki_ @shehbazk @zzeng @PrathapKumar @vaishaakb  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

[#3]

I @itsmezeeshan No it's not recommended to move these files. as your CM server is dependent on it. 

Could you please share the output of the below commands:

# df -h | tee -a /tmp/df.out

# du -hs /* | tee -a /tmp/du.out

[#4]

As per requirement, please find below result against both commands

df -h | tee -a /tmp/df.out
Filesystem Size Used Avail Use% Mounted on
devtmpfs 71G 0 71G 0% /dev
tmpfs 71G 28K 71G 1% /dev/shm
tmpfs 71G 193M 71G 1% /run
tmpfs 71G 0 71G 0% /sys/fs/cgroup
/dev/mapper/rhel-root 19G 6.3G 13G 34% /
/dev/sda1 1.9G 201M 1.7G 11% /boot
/dev/mapper/appvg-lv_app 2.2T 1.4T 779G 65% /apps
/dev/mapper/dumpvg-lv_dump 50G 33M 50G 1% /Cdump
/dev/mapper/rhel-var 19G 17G 2.4G 88% /var
/dev/mapper/logvg-vtmp_lv 20G 33M 20G 1% /var/tmp
/dev/mapper/logvg-log_lv 20G 1.2G 19G 6% /var/log
/dev/mapper/logvg-log_aud_lv 20G 843M 20G 5% /var/log/audit
/dev/mapper/rhel-opt 14G 2.5G 11G 19% /opt
/dev/mapper/rhel-patrol 5.0G 920M 4.1G 18% /opt/patrol
/dev/mapper/rhel-home 19G 15G 4.5G 76% /home
/dev/mapper/logvg-tmp_lv 20G 720M 20G 4% /tmp
cm_processes 71G 174M 71G 1% /run/cloudera-scm-agent/process

du -hs /* | tee -a /tmp/du.out
1.4T /apps
169M /boot
28K /dev
48M /etc
15G /home
3.4G /opt
236K /patch
2.8G /root
367M /run
683M /tmp
2.8G /usr
19G /var

[#5]

@shehbazk your support is required as information already shared with you. 

[#6]

Hello @itsmezeeshan 🙂

Thank you for bringing this to our Community!

Adding to previous post, Here is the storage planning guide for Cloudera Manager you may want to review:

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-storage-space-plann...

Let us know how it goes.

Cheers

V

[#7 · Accepted solution]

Hello @itsmezeeshan, Hope you are doing good. Its not recommended to clear the space from this location. As these the binaries for the Cloudera Manager packages for the following services (cloudera-host-monitor, cloudera-scm-agent, cloudera-scm-eventserver, cloudera-scm-headlamp, cloudera-scm-server,  cloudera-scm-server-db, cloudera-service-monitor)

[root@mprl509 lib]# du -sh cloudera-*
1.3G cloudera-host-monitor
300K cloudera-scm-agent
20M cloudera-scm-eventserver
4.6M cloudera-scm-headlamp
3.0G cloudera-scm-server
272M cloudera-scm-server-db
5.5G cloudera-service-monitor

I would like to recommend you to please increase the disk space to 20 GB More. 

[#8]

In addition as per your last comments, we can see you have a considerable amount of data under your /home directory.

ex: 

15G /home

Could you please check if you can remove some data from it? 

Thanks