This article details the resolution of a startup failure for Knox Gateway roles during a Data Hub upgrade, involving the reconfiguration of a missing knox_pac4j_password.
Symptoms
While upgrading the Data Hub cluster, Knox Gateway services failed to start.
Error message:java.lang.IllegalArgumentException: The variable [${knox_pac4j_password}] does not have a corresponding value.
The upgrade process was halted due to this failure.
Cause
The probable cause of the startup failure was an improperly generated or missingknox_pac4j_passwordduring the upgrade process.
The built-in upgrade handler should have automatically generated the Knox secret, but it did not.
Instructions
To resolve the issue, the following steps were taken:
Verify if an entry forknox_pac4j_passwordexists in the Cloudera Manager database.
Execute SQL query:SELECT * FROM configs WHERE attr LIKE '%knox_pac4j_password%';
Generate a 16-bit UUID random password.
Use OpenSSL:openssl rand -base64 16
Create a JSON file (test.json) with the generated password as follows:
Confirm the Knox service name via Cloudera API call.
Usecurlto fetch service details.
Update the Cloudera Manager configuration with the new password using Cloudera API.
Usecurlwith the PUT method to update the configuration.
Confirm the new value is persisted in the Cloudera Manager database.
Execute SQL query:SELECT * FROM configs WHERE attr LIKE '%knox_pac4j_password%';
Restart or start the Knox service.
Use Cloudera Manager UI or API to restart the service.
Following these steps successfully resolved the issue, allowing the Knox service to start without errors, and the upgrade process was able to continue.