Airflow jobs Monitoring Implementation Steps - Operational Data Engineering Process Monitoring

[#1]

Part of the Visibility and Transparency series this article provides steps for the implementation of Airflow processes or workloads monitoring and alerting system. The framework uses Apache Hive, Python, and BI visualization tools. The features include customized dashboards for insights and alert mechanism.

The attachment to this article details the implementation steps.