What is the Cloudera Production cluster maintenance activities and time frames?

vinodnerella — Sun, 11 Aug 2019 13:09:29 GMT

Community,

Any best practices for the maintenance for production clusters. When we should restart the services generally? - weekly once or monthly once.

Cleaning up log files.

And, also found that yarr logs in HDFS not get deleted according to the retention period. Ref: YARN logs under /tmp/logs/{user.name}/logs not cleared properly

Note: Mainly looking for a cluster in Bank, with regular ingestion using Sqoop and analytics using spark and impala.

Re: What is the Cloudera Production cluster maintenance activities and time frames?

EricL — Sun, 11 Aug 2019 23:42:07 GMT

Hi Vinod,

You should NOT need to restart services on weekly or monthly basis, unless you have scheduled maintenance work like making configurations changes. CDH services are expected to continue their services without the need to restart, unless you face any issues like memory pressure, crashing etc, which will require further investigation.

Regarding YARN aggregation log not cleared properly, please refer to below post and KB:

https://community.cloudera.com/t5/Support-Questions/Yarn-Aggregate-Log-Retention-Setting/m-p/81382
https://my.cloudera.com/knowledge/YARN-logs-under-tmp-logs-user-name-logs-not-cleared-properly?id=75330

Cheers
Eric