Hi Vinod, You should NOT need to restart services on weekly or monthly basis, unless you have scheduled maintenance work like making configurations changes. CDH services are expected to continue their services without the need to restart, unless you face any issues like memory pressure, crashing etc, which will require further investigation. Regarding YARN aggregation log not cleared properly, please refer to below post and KB: https://community.cloudera.com/t5/Support-Questions/Yarn-Aggregate-Log-Retention-Setting/m-p/81382 https://my.cloudera.com/knowledge/YARN-logs-under-tmp-logs-user-name-logs-not-cleared-properly?id=75330 Cheers Eric ... View more

Hi, The purpose of compression is to save space, not speed up query time. Compression actually adds overhead to decompress the data before data can be read, so I would expect the query against compressed data will be slightly slower than uncompressed. So what you see is totally normal to me. Cheers Eric ... View more

Hi, I do not think there is any different. Spark lazily executes statements, so you second 2 jobs version will behave the same way as the first single job, in my opinion. Cheers Eric ... View more