Support Questions

Yong · ‎04-18-2020

The Atlas service was successfully installed and I could see the web UI port is open by using netstat. But I can't connect the Atlas Web UI(503 error), in the /var/log/atlas I didn't find any suspicious information as well. Any ideas why the web UI didn't work?

yacine_ · ‎11-09-2021

I finally managed to resolve this problem with the help of the cloudera support

we have below error in log

---------- WARN - [main:]
~ Failed startup of context o.e.j.w.WebAppContext@6c5945a7{/,file:///opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas/,UNAVAILABLE}
{/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas} 
(WebAppContext:533) java.net.ConnectException: Connexion terminée par expiration du délai d'attente (Connection timed out)

it seems to be a known issue :

https://community.cloudera.com/t5/Customer/quot-HTTP-ERROR-503-Service-Unavailable-URI-quot-when-Atl...

So the solution is :

in each Atlas host

- Delete the following lines from /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas/WEB-INF/web.xml

20 <!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
21 http://java.sun.com/dtd/web-app_2_3.dtd>

- Restart ATLAS

View solution in original post

yacine_ · ‎10-24-2021

I face the same issue, did you managed to resolve it ?

Scharan · ‎10-24-2021

@yacine_ Is Atlas initialization is done, if not can you try Initiazing Atlas

- Navigate to Atlas from the Cloudera Manager UI
- Under actions, the Initialize Atlas option appears. Initiate it; it will create the required resources.
- Once done, verify the Altas UI.

yacine_ · ‎10-25-2021

Hi @Scharan, thank you for the replay,

Yes for sur, I did this.

All Needed kafka topics, hbase tables and Solr collections all are created, needed policies on Ranger too are good.

I even removed atlas and removed all policies, hbase atlas tables, solr atlas collections, kafka atlas topics and reinstalled atlas on a new host, and I get the same error

I cannot neither enable the service Atlas for HBase

there what I have on atlas application.log :

2021-10-24 21:03:45,634 INFO - [main:] ~ Attempting to retrieve password for key password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-10-24 21:03:45,638 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-10-24 21:03:45,638 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-10-24 21:03:45,639 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-10-24 21:03:45,639 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-10-24 21:03:45,639 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-10-24 21:03:45,740 INFO - [main:] ~ Attempting to retrieve password for key truststore.password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-10-24 21:03:45,745 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-10-24 21:03:45,746 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546354754-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-10-24 21:03:45,746 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-10-24 21:03:45,746 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-10-24 21:03:45,746 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-10-24 21:05:53,494 DEBUG - [main:] ~ &lt;== AtlasErrorCode.getMessage([]) (AtlasErrorCode:247)
2021-10-24 21:05:53,494 DEBUG - [main:] ~ ==&gt; AtlasErrorCode.getMessage([]): EmbeddedServer.Start: failed! (AtlasErrorCode:254)

Scharan · ‎10-25-2021

enable debug on atlas logs and see if their are any errors

yacine_ · ‎11-06-2021

Hi @Scharan I reinstalled atlas and I still getting the same Error,

There the trace logs of atlas (application.log)

2021-11-06 18:45:59,180 INFO - [main:] ~ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Atlas:216)
2021-11-06 18:45:59,180 INFO - [main:] ~ Server starting with TLS ? true on port 31443 (Atlas:217)
2021-11-06 18:45:59,180 INFO - [main:] ~ <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< (Atlas:218)
2021-11-06 18:45:59,227 INFO - [main:] ~ Attempting to retrieve password for key keystore.password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-11-06 18:45:59,248 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-11-06 18:45:59,249 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-11-06 18:45:59,249 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-11-06 18:45:59,249 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-11-06 18:45:59,249 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-11-06 18:45:59,450 INFO - [main:] ~ Attempting to retrieve password for key truststore.password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-11-06 18:45:59,457 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-11-06 18:45:59,457 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-11-06 18:45:59,458 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-11-06 18:45:59,458 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-11-06 18:45:59,458 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-11-06 18:45:59,621 INFO - [main:] ~ Attempting to retrieve password for key keystore.password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-11-06 18:45:59,628 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-11-06 18:45:59,628 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-11-06 18:45:59,629 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-11-06 18:45:59,629 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-11-06 18:45:59,629 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-11-06 18:45:59,752 INFO - [main:] ~ Attempting to retrieve password for key password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-11-06 18:45:59,756 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-11-06 18:45:59,756 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-11-06 18:45:59,756 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-11-06 18:45:59,757 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-11-06 18:45:59,757 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-11-06 18:45:59,856 INFO - [main:] ~ Attempting to retrieve password for key truststore.password from cert.stores.credential.provider.path configured credential provider path localjceks://file//var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (SecurityUtil:63)
2021-11-06 18:45:59,861 DEBUG - [main:] ~ backing jks path initialized to file:/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks (AbstractJavaKeyStoreProvider:169)
2021-11-06 18:45:59,861 DEBUG - [main:] ~ initialized local file as '/var/run/cloudera-scm-agent/process/1546358895-atlas-ATLAS_SERVER/conf/atlascreds.localjceks'. (LocalKeyStoreProvider:123)
2021-11-06 18:45:59,861 DEBUG - [main:] ~ the local file exists and is size 1481 (LocalKeyStoreProvider:125)
2021-11-06 18:45:59,861 TRACE - [main:] ~ we can read the local file. (LocalKeyStoreProvider:128)
2021-11-06 18:45:59,861 TRACE - [main:] ~ we can write the local file. (LocalKeyStoreProvider:131)
2021-11-06 18:48:07,589 DEBUG - [main:] ~ <== AtlasErrorCode.getMessage([]) (AtlasErrorCode:247)
2021-11-06 18:48:07,590 DEBUG - [main:] ~ ==> AtlasErrorCode.getMessage([]): EmbeddedServer.Start: failed! (AtlasErrorCode:254)

Scharan · ‎11-06-2021

@yacine_ Can you confirm the cdp version

yacine_ · ‎11-07-2021

@Scharan CDP 7.1.6

By the way, when I reinstalled Atlas, I do Initialize Atlas from CM, I can see this error in the log, (I don't know if it's related to my problem or not )

+ ATLAS_CERT_STORES_CREDENTIAL_PROVIDER_PATH=localjceks://file//var/run/cloudera-scm-agent/process/1546359070-atlas-ATLAS_SERVER-InitializeAtlasRole/conf/atlascreds.localjceks
+ set +x
java.lang.IllegalArgumentException: Empty key
 at java.base/javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:95)
 at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.innerSetCredential(AbstractJavaKeyStoreProvider.java:272).   at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.createCredentialEntry(AbstractJavaKeyStoreProvider.java:237)
 at org.apache.hadoop.security.alias.CredentialShell$CreateCommand.execute(CredentialShell.java:354) at org.apache.hadoop.tools.CommandShell.run(CommandShell.java:72)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
 at org.apache.hadoop.security.alias.CredentialShell.main(CredentialShell.java:437)

yacine_ · ‎11-09-2021

I finally managed to resolve this problem with the help of the cloudera support

we have below error in log

---------- WARN - [main:]
~ Failed startup of context o.e.j.w.WebAppContext@6c5945a7{/,file:///opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas/,UNAVAILABLE}
{/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas} 
(WebAppContext:533) java.net.ConnectException: Connexion terminée par expiration du délai d'attente (Connection timed out)

it seems to be a known issue :

https://community.cloudera.com/t5/Customer/quot-HTTP-ERROR-503-Service-Unavailable-URI-quot-when-Atl...

So the solution is :

in each Atlas host

- Delete the following lines from /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/atlas/server/webapp/atlas/WEB-INF/web.xml

20 <!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
21 http://java.sun.com/dtd/web-app_2_3.dtd>

- Restart ATLAS

smdas · ‎11-10-2021

Hello @yacine_

Thanks for sharing the Solution along with the Root Cause as well. We shall mark the Post as Solved for now to ensure fellow Community Users can use the Solution as well.

Regards, Smarak

Cloudera Community

Support Questions

CDP7 can't access Atlas Web UI