Because spark action in oozie is not supported in HDP 2.3.x and HDP 2.4.0, there is no workaround especially in kerberos environment. We can use either java action or shell action to launch spark job in oozie workflow. In this article, we will discuss how to use oozie shell action to run a spark job in kerberos environment.
1. Spark client is installed on every host where nodemanager is running. This is because we have no control over which node the
2. Optionally, if the spark job need to interact with hbase cluster, hbase client need to be installed on every host as well.
1. Create a shell script with the spark-submit command. For example, in the script.sh: