Hello @RajeshBodolla    It seems like the actual exception is as below   Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]   Could you please share the complete stack? it seems it is somewhere not able to resolve the principal ... View more

Hello @regeamor   Thank you for posting the query with us.   Basically when you enable Dynamic allocation it gracefully remove the Idle containers which were idle for (60s as default) But when you lower the value will remove the executors frequently depending upon the executor's usage (just like the tasks getting allocated to those executors and the amount of tasks required)   https://spark.apache.org/docs/latest/job-scheduling.html#graceful-decommission-of-executors   Also, the above said behaviour is upto Spark (and yarn container pre-emption wont be aware of it) ... View more

Hello @yasmin,   Thanks for reaching us !   As per your reply it says that user who has triggered job, his user id is not listed on that particular node ? Is my understanding correct? ==> Yes you're right !   And your query regarding job slowness, we should consider the factors you mentioned along with below messages as well   ~~~ ExecutorLostFailure (executor 1480 exited caused by one of the running tasks) Reason: Stale executor after cluster manager re-registered.   I had 438 failures on 869 taks, that is a huge rate, another part has 873 out of 1236 ~~~   Here it seems the executors are getting lost and as a results tasks were dying.   Could you please check the yarn logs for the application (by using #yarn logs command in my previous reply) and see if there is any errors in executor logs. This will help us to see if there is any spcific reason for executor failures   Are you running  job in spark-client mode or cluster mode?   ... View more

Hello @yasmin   Thanks for posting your query.   From the error message attached I see the Nodemanager node is complaining that the user ID is not present on the node. Please check if the username is present on the node (where AM is running) or if you are using AD/LDAP just make sure the node is able to resolve the particular username    You can also check more on container logs, by running    #yarn logs -applicationId <applicationID> -appOwner <username_who_triggered_job>    ... View more

Hello @upasana   Could you please share the complete log / exception trace, so that we can see from where you are getting that error   ... View more

Please review the below Email discussion thread as well !   https://groups.google.com/a/cloudera.org/forum/#!topic/hue-user/fcRM3YiqAAA ... View more

Hello @upasana   Thanks for posting your query!   I see you are facing issue while calling the Livy's rest API for submitting the job and it is complaining on "java.lang.NoClassDefFoundError"   I would like to clarify, the error message "java.lang.NoClassDefFoundError" whilch you are getting is from the application (which holds the REST API call) or from the actual user job ?   If you are getting this error from the user application (which you are invoking through REST API) you might need to add the jar in executors/Driver's classpath   If you are getting above error in your application (which invokes the REST call) you might need to add the jar in your application's classpath   Please correct me if my above understanding is wrong  ... View more

Hello @ursash   Glad to know your issue got relsoved !   Yes, the API calls "GET /jobs/ failed" might be suffered because of your conflict   ... View more