02-23-2015 07:08 AM
I tried editing the /etc/spark/conf/log4j.properties file and changed all log levels to WARN. Still when viewing application logs from YARN resource manager I see INFO messages.
02-23-2015 07:37 AM
Yes, you should not edit these files manually. Have a look at the YARN configuration in CM. You should quickly find the Logs tab for the various components like ResourceManager. You can configure log levels there.
02-24-2015 02:44 AM
Yes, after loggin in to Cloudera Manager, I clicked on YARN service, then clicked on the Configuration tab. In the search field I entered logging threshold and changed them all to WARN (JobHistory Server Logging Threshold, NodeManager Logging Threshold, NodeManager Logging Threshold,ResourceManager Logging Threshold)