Hello @wert_1311  This is an Old Post, yet I am answering to ensure the Post can be used for future reference.  Anytime your Team observe the above Issue, Capture the CDE Diagnostics Bundle covering the Timeframe of Issue (Example Is 2 days In The Screenshot). Next, Restart the Airflow Scheduler Pod by Deleting the Airflow Scheduler Pod, upon which the Airflow Scheduler Pod would be Recreated implicitly. Next, Engage Cloudera Support with the CDE Diagnostics Bundle captured above. Additionally, CDE Airflow has been significantly Scaled-Test in recent releases of CDE & your Team should consider Upgrading CDE to Latest Version as soon as possible. - Smarak [1] https://docs.cloudera.com/data-engineering/cloud/troubleshooting/topics/cde-download-diagnostic-bundle.html  ... View more

Hello @Hae  AFAIK, Data Connections is a Public Cloud Concept & isn't available in Private Cloud yet. In Public Cloud, [1] shows the Steps to configure Data Connections, which allows you to access the HMS of the DataLake (Unified HMS Source For The Environment). In Private Cloud, You may use the [2] to use Spark on CML. The same has Example on using Spark-On-Yarn on Base Cluster as well as Spark-On-Kubernetes on CML. - Smarak [1] https://docs.cloudera.com/machine-learning/cloud/mlde/topics/ml-mlde-spark-data-connection.html  [2] https://docs.cloudera.com/machine-learning/1.5.2/spark/topics/ml-apache-spark-overview.html  ... View more

Hello @RamaClouder  Hope you have Synced with Cloudera Sales Team [0] & have no further ask. To Sum up of the Post, Cloudera offers Unified Governance within the Platform on Public, Private & Hybrid Cloud Setup. The Link [1] has relevant Doc/Videos Link for further Understanding.  If you have any further ask, Feel free to post in our Community & We shall get back to you accordingly.  - Smarak [0] https://www.cloudera.com/contact-sales.html [1] https://www.cloudera.com/products/cloudera-data-platform/sdx.html  ... View more

Hello @Rekha35    Thanks for using Cloudera Community. First, I shall answer the Q from Cloudera's perspective. Cloudera Data Engineering offering (CDE) offers Airflow built-in without any manual intervention. CDE is available on Cloudera Public Cloud & Private Cloud offering. The integration with other Platform Services is Seamless & offers an immersive experience to End-Users looking to explore Airflow. Note that your Team can integrate with an External Airflow [2] as well.    Considering your ask was with checking out Airflow, You may try out [3], wherein the "Astronomer Apache Airflow Fundamentals Certification" offers the best starting point (Including Installing Locally). The Course is led by the foremost Airflow Instructor Marc Lamberti & helps anyone delve into Airflow seamlessly. Additionally, You asked about checking Airflow on Cloud owing to having issues with deploying the same locally. Each Cloud Provider (AWS, Azure, GCP) offers a Managed Service for Airflow.    Hope your Q has been addressed by us. Feel free to share any further ask.    Regards, Smarak     [1] https://docs.cloudera.com/data-engineering/cloud/orchestrate-workflows/topics/cde-airflow-dag-pipeline.html  [2] https://docs.cloudera.com/data-engineering/cloud/orchestrate-workflows/topics/cde-airflow-provider.html?  [3] https://www.astronomer.io/certification/  ... View more