Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Apache Airflow installation in CDH 5.13 VM

avatar
Contributor

Hi All,

 

Could you please someone please help me how to install the apache airflow parcel in the cloudera VM.Our project moving towards airflow orchestration.please help me on this

 

Is there any way we can do this in cloudera quickstart VM 5.13?.please share the thoughts on this.

 

Thanks

Anbu

3 REPLIES 3

avatar

It is a bit of a late reply - haven't tried it myself yet but looks promising - https://blog.clairvoyantsoft.com/apache-airflow-csd-ac5b145d5e2d

avatar
New Contributor

The installation seems promising.. but how do you create an airflow DAG, (like creating the workflow and job.properties for Oozie). I think you need an operator for hadoop job, but I don't think there is one yet.

avatar

I managed to integrate Airflow with redis into Cloudera Manager. To run custom DAG, they need to be uploaded to the airflow dag folder on the node, where airflow scheduler and workers are dunning