Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Airflow is failing to start in cloudera with issue Execute command Upgrade Airflow DB on service Airflow Could not create process: null

avatar
New Contributor

Hello all,

We are having an issue while adding airflow service to the CDH 5 cluster.

Problem: Executing the command "Upgrade Airflow DB" on the Airflow service in a Cloudera cluster on GCP (CentOS 7, CDH 5.16.2, Airflow 1.10.1) results in an error: "Could not create process: null". The attempt to upgrade the Airflow database fails while adding airflow service to the CDH cluster.

Context:

  • The same infrastructure and parcel configurations have been successfully used on 6-7 on-premises clusters.
  • Logs are empty, and the scm-server log provides the same error message, indicating a failure to create the process.

Environment:

  • Operating System: CentOS 7
  • CDH Version: 5.16.2
  • Airflow Version: 1.10.1

Steps Taken:

  • Added the Airflow parcels and csd in the proper location with cloudera-scm owner.
  • Added the Airflow service.
  • Created Mysql DB for airflow.
  • Added Mysql lib/client on the all hosts.
  • Added Redis server to manage the airflow in cloudera.
  • Checked logs for detailed error information, but logs are empty.
  • Verified the successful deployment of Airflow on multiple on-premises clusters with the same configuration.

Issue Impact:

  • Unable to add the Airflow service on the Cloudera cluster in GCP, affecting the stability and functionality of Airflow.

Request for Assistance:

  • Seeking community insights and assistance in identifying the root cause of the issue.
  • Open to recommendations, troubleshooting steps, or insights from the community to resolve the "Could not create process: null" error during the Airflow DB upgrade.

Additional Information:

  • No recent changes in the configuration that could explain the failure.
  • SCM-server logs are not providing additional information.

Environment Details:

  • Cloudera Manager: Version 5.16.2
  • CDH: Version 5.16.2
  • Airflow: Version 1.10.1
  • Operating System: CentOS 7

Your collaboration and expertise are highly appreciated in resolving this issue.

1 REPLY 1

avatar
Community Manager

@Mdismaik Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Airflow experts @SanthoshGowda @smdas  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: