Without reviewing the logs, If I can guess on the issue, If you are submitting the Spark job from Oozie Shell action, then I would suspect the problem is with Oozie's behavior of setting the environment variable HADOOP_CONF_DIR behind the scene. There is an internal Jira that tracks this behavior/work. The KB [1] explains a bit on this (even though it is reported for the hive-site.xml, I think it may influence the HBase client conf as well).   Try working around the problem by following the instructions on the KB [1] and see if it helps.   Thanks Lingesh [1]: https://community.cloudera.com/t5/Customer/How-to-run-a-Spark2-job-using-Oozie-Shell-Action-which/ta-p/73185 ... View more

Hi Alex,   Did u checked in Oozie Configuration or in oozie logs like whether the Event logs are writing in some other path apart from the path that was configured in CM?   Thanks AKR ... View more