Created on 04-26-2019 02:16 AM - edited 09-16-2022 07:20 AM
Created 04-28-2019 12:12 AM
You can use the Spark Action in Oozie to submit any spark applications:
https://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SparkActionExtension.html#Spark_Action
If you are more familar with spark-submit tool, you can try to use oozie shell action as well:
https://archive.cloudera.com/cdh5/cdh/5/oozie/DG_ShellActionExtension.html
You may need to make sure the spark gateway role is deployed on the oozie server and node manager nodes, so that the runtime env always have the depencies available.
Created 04-28-2019 12:12 AM
You can use the Spark Action in Oozie to submit any spark applications:
https://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SparkActionExtension.html#Spark_Action
If you are more familar with spark-submit tool, you can try to use oozie shell action as well:
https://archive.cloudera.com/cdh5/cdh/5/oozie/DG_ShellActionExtension.html
You may need to make sure the spark gateway role is deployed on the oozie server and node manager nodes, so that the runtime env always have the depencies available.