Support Questions
Find answers, ask questions, and share your expertise

Different frameworks to automate Hadoop jobs

Different frameworks to automate Hadoop jobs

Hello

What are the different frameworks to automate Hadoop jobs? Which is the best?

Thanks

2 REPLIES 2
Highlighted

Re: Different frameworks to automate Hadoop jobs

Super Collaborator

Your open source options include

Oozie (included with HDP), Luigi, Airflow, and Azkaban.

https://www.bizety.com/2017/06/05/open-source-data-pipeline-luigi-vs-azkaban-vs-oozie-vs-airflow/

NiFi is also capable of scheduling tasks.

An enterprise option includes Stonebranch UAC

"Best" is relative to your needs and who will support it

Highlighted

Re: Different frameworks to automate Hadoop jobs

New Contributor