Yarn/Spark jobs Monitoring Implementation Steps - Operational Data Engineering Process Monitoring

Part of the Visibility and Transparency series this article provides steps for the implementation of Yarn/Spark processes or workloads monitoring and alerting system. The framework uses Apache Hive, Apache NiFi, and BI visualization tools. The features include customized visualization dashboards for insights and alert mechanisms.

The attachment to this article details the implementation steps.