Skip to main content
wert_1311
Gold ⭐️
October 9, 2023
Solved

Airflow scheduler does not appear to be running.

We are seeing an error / alert and unable to schedule / run jobs in CDE. We are on CDP - PC

Annotation 2023-10-09 202100.jpg

Any guidance  in resolving this 

Tks 

Wert

smdas
smdasAnswer
Gold ⭐️⭐️
April 15, 2024

Hello @wert_1311 

This is an Old Post, yet I am answering to ensure the Post can be used for future reference. 

Anytime your Team observe the above Issue, Capture the CDE Diagnostics Bundle covering the Timeframe of Issue (Example Is 2 days In The Screenshot). Next, Restart the Airflow Scheduler Pod by Deleting the Airflow Scheduler Pod, upon which the Airflow Scheduler Pod would be Recreated implicitly. Next, Engage Cloudera Support with the CDE Diagnostics Bundle captured above. Additionally, CDE Airflow has been significantly Scaled-Test in recent releases of CDE & your Team should consider Upgrading CDE to Latest Version as soon as possible.

- Smarak

[1] https://docs.cloudera.com/data-engineering/cloud/troubleshooting/topics/cde-download-diagnostic-bundle.html 

1 reply

smdas
Gold ⭐️⭐️
smdasAnswer
Gold ⭐️⭐️
April 15, 2024

Hello @wert_1311 

This is an Old Post, yet I am answering to ensure the Post can be used for future reference. 

Anytime your Team observe the above Issue, Capture the CDE Diagnostics Bundle covering the Timeframe of Issue (Example Is 2 days In The Screenshot). Next, Restart the Airflow Scheduler Pod by Deleting the Airflow Scheduler Pod, upon which the Airflow Scheduler Pod would be Recreated implicitly. Next, Engage Cloudera Support with the CDE Diagnostics Bundle captured above. Additionally, CDE Airflow has been significantly Scaled-Test in recent releases of CDE & your Team should consider Upgrading CDE to Latest Version as soon as possible.

- Smarak

[1] https://docs.cloudera.com/data-engineering/cloud/troubleshooting/topics/cde-download-diagnostic-bundle.html 

Translate