Support Questions
Find answers, ask questions, and share your expertise

Apache Airflow installation in CDH 5.13 VM

Apache Airflow installation in CDH 5.13 VM

Hi All,

 

Could you please someone please help me how to install the apache airflow parcel in the cloudera VM.Our project moving towards airflow orchestration.please help me on this

 

Is there any way we can do this in cloudera quickstart VM 5.13?.please share the thoughts on this.

 

Thanks

Anbu

3 REPLIES 3

Re: Apache Airflow installation in CDH 5.13 VM

It is a bit of a late reply - haven't tried it myself yet but looks promising - https://blog.clairvoyantsoft.com/apache-airflow-csd-ac5b145d5e2d

Re: Apache Airflow installation in CDH 5.13 VM

New Contributor

The installation seems promising.. but how do you create an airflow DAG, (like creating the workflow and job.properties for Oozie). I think you need an operator for hadoop job, but I don't think there is one yet.

Re: Apache Airflow installation in CDH 5.13 VM

I managed to integrate Airflow with redis into Cloudera Manager. To run custom DAG, they need to be uploaded to the airflow dag folder on the node, where airflow scheduler and workers are dunning