Created 12-13-2017 09:28 AM
Hello
What are the different frameworks to automate Hadoop jobs? Which is the best?
Thanks
Created 12-14-2017 04:25 PM
Your open source options include
Oozie (included with HDP), Luigi, Airflow, and Azkaban.
https://www.bizety.com/2017/06/05/open-source-data-pipeline-luigi-vs-azkaban-vs-oozie-vs-airflow/
NiFi is also capable of scheduling tasks.
An enterprise option includes Stonebranch UAC
"Best" is relative to your needs and who will support it
Created 07-30-2018 12:48 PM
Another recent comparison article: https://xunnanxu.github.io/2018/04/13/Workflow-Processing-Engine-Overview-2018-Airflow-vs-Azkaban-vs...