Yarn QueueUI not Loading

[#1]

Hello,

I am unable to launch Yarn Queue Manager Web UI and get error - Failed to load the Yarn Queue Manager UI. Reload the UI after fixing the issue. Reason - No active RM Found.

We have 2 RM (Active/ Standby) yet facing this error. 

Reques any help / guidance to resolve this.

CM 7.6.1

Cloudera Runtime - 7.1.7

[#2]

Hello @Amn_468 

1) The quick fix might be to restart both RM and YQM and see if that clears the issue.

If you still face the same issue, then

2) Check if you are using TLSL for CM and YARN. If yes, then configure TLS for Yarn Queue Manager as well [1].

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/yarn-security/topics/yarn-configure-tls-ssl-y...

3) If Kerberos is enabled for YARN, then enable it for the YARN queue manager. Could you please verify if Kerberos security is configured correctly using the below doc [1]:
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/security-kerberos-authentication/topics/cm-se...

If you still face the same issue, then please upload the below queue manager log files from the Queue manager host for the issue time stamp:-
*queuemanager-webapp.log
*config-service.log
*config-service-requestlog.log
location : /var/log/yarn/queuemanager/

Please upload ResourceManager log files for the issue time-stamp - location : /var/log/hadoop-yarn/

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

[#3]

Hi @Babasaheb 

Thanks for your reply, I have tried step 1, but it did not help. 

We have Auto TLS Enabled:

[root@MYSERVER ~]# grep use_tls /etc/cloudera-scm-agent/config.ini
use_tls=1

The error in queuemanager-webapp.log is :

2022-11-24 20:16:53,871 ERROR com.cloudera.cpx.server.api.utils.HttpClientInstance: https://MYSERVER.com:8090/ws/v1/cluster service is not available ! Error:
com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Unsure why the certificates are not getting picked up.

Again appreciate all your assistance.

[#4]

Hi @Babasaheb 

We have added certificates in java trust store (cacerts) for all the servers. Yarn QM UI does open, but not in tls mode. We have enabled auto tls, and the yarn queue manager / UI does not open with tls enabled for it. Attached are the error and role log snips

Role Log

Stderror

Thanks

[#5 · Accepted solution]

@Amn_468 

Thank you for sharing the logs.

We can see that you are getting the following error.

Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -1

As per the above exception we have checked the code of WebappProperties and it's failing while parsing the configs with this code:

=============

 } else if (val.contains("#")) {

  val = val.substring(0, val.indexOf(" "));

=============

There is # in your SSL password hence it's failing. We have replicated this issue in the test lab and the queue manager was failing to start with the same error. 

You should change the password of keystore and truststore and make sure that it will not have # in it or remove SSL settings temporarily.

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

[#6]

@Babasaheb  Appreciate all you assistance