Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

How can I execute Spark jobs periodically?



I currently use spark submit to execute an SBT created jar.

How can I automate the process in order to let it start periodically (i.e. every night or every few hours?)

thank you


Rising Star

@Ir Mar Starting with HDP 2.6, you can use Workflow Designer to design and schedule work flows, including Spark jobs. Documentation is at Alternatively, you can use Oozie to schedule Spark workflows, and details around that can be found at Hope this helps, and please remember to upvote / accept the answer if you found this useful.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.