How can I execute Spark jobs periodically?

[#1]

Hello,

I currently use spark submit to execute an SBT created jar.

How can I automate the process in order to let it start periodically (i.e. every night or every few hours?)

thank you

[#2]

@Ir Mar Starting with HDP 2.6, you can use Workflow Designer to design and schedule work flows, including Spark jobs. Documentation is at https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_workflow-management/content/ch_wfm_basic.... Alternatively, you can use Oozie to schedule Spark workflows, and details around that can be found at https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_spark-component-guide/content/ch_oozie-s.... Hope this helps, and please remember to upvote / accept the answer if you found this useful.