Member since
06-14-2021
1
Post
0
Kudos Received
0
Solutions
01-18-2024
08:41 AM
Hello all, We are having an issue while adding airflow service to the CDH 5 cluster. Problem: Executing the command "Upgrade Airflow DB" on the Airflow service in a Cloudera cluster on GCP (CentOS 7, CDH 5.16.2, Airflow 1.10.1) results in an error: "Could not create process: null". The attempt to upgrade the Airflow database fails while adding airflow service to the CDH cluster. Context: The same infrastructure and parcel configurations have been successfully used on 6-7 on-premises clusters. Logs are empty, and the scm-server log provides the same error message, indicating a failure to create the process. Environment: Operating System: CentOS 7 CDH Version: 5.16.2 Airflow Version: 1.10.1 Steps Taken: Added the Airflow parcels and csd in the proper location with cloudera-scm owner. Added the Airflow service. Created Mysql DB for airflow. Added Mysql lib/client on the all hosts. Added Redis server to manage the airflow in cloudera. Checked logs for detailed error information, but logs are empty. Verified the successful deployment of Airflow on multiple on-premises clusters with the same configuration. Issue Impact: Unable to add the Airflow service on the Cloudera cluster in GCP, affecting the stability and functionality of Airflow. Request for Assistance: Seeking community insights and assistance in identifying the root cause of the issue. Open to recommendations, troubleshooting steps, or insights from the community to resolve the "Could not create process: null" error during the Airflow DB upgrade. Additional Information: No recent changes in the configuration that could explain the failure. SCM-server logs are not providing additional information. Environment Details: Cloudera Manager: Version 5.16.2 CDH: Version 5.16.2 Airflow: Version 1.10.1 Operating System: CentOS 7 Your collaboration and expertise are highly appreciated in resolving this issue.
... View more
Labels:
- Labels:
-
Apache Airflow