Support Questions
Find answers, ask questions, and share your expertise

AMBARI 2.7.3 UI not progressing to next step during cluster deploy

Explorer

Hello Experts,

 

I'm trying to install Open source AMBARI 2.7.3 in SLES 12 SP2 and SP3 environments.
I was able to build Ambari 2.7.4 source successfully as per the steps mentioned in Installation Guide for Ambari 2.7.4, Reference - https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.4


After build, i was able to successfully install and start Ambari server.

After logging to Ambari UI from browser i wanted to deploy the cluster using Ambari UI.
Current issue im facing is, when i click next after inputting name of the cluster as shown in screenshot, AMBARI 2.7.3 UI not porgressing to next step during cluster deploy
It gets spinning for long time forever and stays in Get started Page itself without any progress

Please suggest fix.

 

I already tried with create 'Reset UI' button in Ambari's experimental page and retired but no luck
I also tried with multiple browsers as well but same issues.
Also, there is no broken HDP stacks.All fine

Please help to fix this issue.

 

Attached the screenshot for reference

AMBARI UI not progressing.PNG
Thanks & Regards,
Chethan

4 REPLIES 4

Contributor

Please provide more info from ambari logs. One thing is to check your backend db connection.

Explorer

1) From Ambari-server logs i can see following error/exception/warning:

I have also attached the full log 

 

2020-08-10 06:15:48,174 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:94 - Loading latest URL info from http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.3.GlusterFS,HDP-2.3.ECS,HDP-2.6
2020-08-10 06:15:48,174 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:94 - Loading latest URL info from http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.5,HDP-2.3,HDP-2.1.GlusterFS,HDP-2.4,HDP-2.1,HDP-2.2,HDP-2.0
2020-08-10 06:15:48,339 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:110 - Loaded URI http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.5,HDP-2.3,HDP-2.1.GlusterFS,HDP-2.4,HDP-2.1,HDP-2.2,HDP-2.0 in 167ms
2020-08-10 06:15:48,943 ERROR [Stack Version Loading Thread] URLStreamProvider:245 - Received HTTP 403 response from URL: http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json
2020-08-10 06:15:48,943 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:107 - Could not load the URI from http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json, stack defaults will be used
2020-08-10 06:15:48,943 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:110 - Loaded URI http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.3.GlusterFS,HDP-2.3.ECS,HDP-2.6 in 772ms
2020-08-10 06:15:48,944 INFO [main] StackContext:180 - Loaded urlinfo in 777ms
2020-08-10 06:15:48,945 ERROR [main] StackContext:189 - Could not load repo results
java.io.IOException: Server returned HTTP response code: 403 for URL: http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1944)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1514)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at org.apache.ambari.server.state.stack.RepoUrlInfoCallable.call(RepoUrlInfoCallable.java:97)
at org.apache.ambari.server.state.stack.RepoUrlInfoCallable.call(RepoUrlInfoCallable.java:50)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://s3.amazonaws.com/dev.hortonworks.com/HDP/hdp_urlinfo.json
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
... 8 more

 

 

2020-08-10 06:15:50,050 INFO [main] PasswordUtils:175 - Credential provider creation failed
org.apache.ambari.server.AmbariException: Master key initialization failed.
at org.apache.ambari.server.security.encryption.CredentialProvider.<init>(CredentialProvider.java:63)
at org.apache.ambari.server.utils.PasswordUtils.loadCredentialProvider(PasswordUtils.java:173)
at org.apache.ambari.server.utils.PasswordUtils.readPasswordFromStore(PasswordUtils.java:148)
at org.apache.ambari.server.configuration.ComponentSSLConfiguration.getPassword(ComponentSSLConfiguration.java:121)
at org.apache.ambari.server.configuration.ComponentSSLConfiguration.init(ComponentSSLConfiguration.java:63)
at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1107)
2020-08-10 06:15:50,425 INFO [main] AmbariManagementControllerImpl:427 - Initializing the AmbariManagementControllerImpl
2020-08-10 06:15:50,508 INFO [main] SingleFileWatch:78 - Starting SingleFileWatcher:ambari.properties
2020-08-10 06:15:50,512 INFO [main] AmbariServer:786 - Jetty is configuring ambari-client-thread with 2 reserved acceptors/selectors and a total pool size of 25 for 8 processors.
2020-08-10 06:15:50,539 INFO [main] AmbariServer:786 - Jetty is configuring qtp-ambari-agent with 4 reserved acceptors/selectors and a total pool size of 25 for 8 processors.
2020-08-10 06:15:50,614 INFO [main] ClustersImpl:281 - Initializing cluster and host data.
2020-08-10 06:15:50,653 INFO [main] ClassPathXmlApplicationContext:583 - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@51df2a41: startup date [Mon Aug 10 06:15:50 CEST 2020]; root of context hierarchy
2020-08-10 06:15:50,908 INFO [main] CertificateManager:75 - Initialization of root certificate
2020-08-10 06:15:50,908 INFO [main] CertificateManager:77 - Certificate exists:true
2020-08-10 06:15:50,971 INFO [main] ViewRegistry:534 - Triggering loading of [ALL] views
2020-08-10 06:15:50,984 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/ambari-admin-2.7.3.0.0.jar.
2020-08-10 06:15:51,092 INFO [main] ViewRegistry:1850 - View deployed: ADMIN_VIEW{2.7.3.0}.
2020-08-10 06:15:51,128 INFO [main] HeartbeatProcessor:160 - **** Starting heartbeats processing threads ****
2020-08-10 06:15:51,130 INFO [main] AmbariServer:496 - ********** Started Heartbeat handler **********
2020-08-10 06:15:51,131 INFO [main] AmbariServer:535 - ********* Initializing Clusters **********
2020-08-10 06:15:51,131 INFO [main] AmbariServer:541 - ********* Current Clusters State *********
2020-08-10 06:15:51,132 INFO [main] AmbariServer:542 -
2020-08-10 06:15:51,132 INFO [main] AmbariServer:544 - ********* Reconciling Alert Definitions **********
2020-08-10 06:15:51,132 INFO [main] AmbariServer:547 - ********* Initializing ActionManager **********
2020-08-10 06:15:51,132 INFO [main] AmbariServer:550 - ********* Initializing Controller **********
2020-08-10 06:15:51,132 INFO [main] AmbariServer:554 - ********* Initializing Scheduled Request Manager **********
2020-08-10 06:15:51,397 INFO [main] ContextLoader:304 - Root WebApplicationContext: initialization started
2020-08-10 06:15:51,398 INFO [main] AnnotationConfigWebApplicationContext:583 - Refreshing Root WebApplicationContext: startup date [Mon Aug 10 06:15:51 CEST 2020]; parent: org.springframework.context.support.ClassPathXmlApplicationContext@51df2a41
2020-08-10 06:15:51,425 INFO [main] AnnotationConfigWebApplicationContext:208 - Registering annotated classes: [class org.apache.ambari.server.configuration.spring.ApiSecurityConfig]
2020-08-10 06:15:51,961 INFO [main] AutowiredAnnotationBeanPostProcessor:155 - JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2020-08-10 06:15:52,776 INFO [main] DefaultSecurityFilterChain:43 - Creating filter chain: org.springframework.security.web.util.matcher.AnyRequestMatcher@1, [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@6292c63e, org.springframework.security.web.context.SecurityContextPersistenceFilter@232b3b4c, org.springframework.security.web.header.HeaderWriterFilter@6fdd3382, org.springframework.security.web.authentication.logout.LogoutFilter@13dc383b, org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter@65bb4cb9, org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter@7b33deed, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@39296cef, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@64e3bc2, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@7fd32c56, org.springframework.security.web.session.SessionManagementFilter@4fea095b, org.springframework.security.web.access.ExceptionTranslationFilter@58e7f930, org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter@b04a6a4, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@472dbaf5]
2020-08-10 06:15:52,846 INFO [main] ContextLoader:344 - Root WebApplicationContext: initialization completed in 1449 ms
2020-08-10 06:15:52,847 WARN [main] DeprecationWarning:43 - Using @Deprecated Class org.eclipse.jetty.servlets.GzipFilter
2020-08-10 06:15:52,847 WARN [main] DeprecationWarning:43 - Using @Deprecated Class org.eclipse.jetty.servlets.GzipFilter
2020-08-10 06:15:52,847 WARN [main] GzipFilter:45 - GzipFilter is deprecated. Use GzipHandler
2020-08-10 06:15:52,847 WARN [main] GzipFilter:45 - GzipFilter is deprecated. Use GzipHandler
2020-08-10 06:15:52,847 INFO [main] ViewThrottleFilter:123 - Ambari Views will be able to utilize 12 concurrent REST API threads
2020-08-10 06:15:52,865 INFO [main] PackagesResourceConfig:101 - Scanning for root resource and provider classes in the packages:
org.apache.ambari.server.api.rest
org.apache.ambari.server.api.services
org.apache.ambari.eventdb.webservice
org.apache.ambari.server.api
2020-08-10 06:15:55,283 INFO [main] ScanningResourceConfig:153 - Root resource classes found:
class org.apache.ambari.server.api.services.views.ViewService
class org.apache.ambari.server.api.services.users.ActiveWidgetLayoutService
class org.apache.ambari.server.api.services.RootServiceService
class org.apache.ambari.server.api.services.HostService
class org.apache.ambari.server.api.services.RemoteClustersService
class org.apache.ambari.server.api.services.ValidationService
class org.apache.ambari.server.api.services.LogoutService
class org.apache.ambari.server.api.services.VersionDefinitionService
class org.apache.ambari.server.api.services.groups.GroupService
class org.apache.ambari.server.api.services.FeedService
class org.apache.ambari.server.api.services.views.ViewPrivilegeService
class org.apache.ambari.server.api.services.users.UserAuthenticationSourceService
class org.apache.ambari.server.api.services.InstanceService
class org.apache.ambari.server.api.services.ClusterService
class org.apache.ambari.server.api.services.AmbariPrivilegeService
class org.apache.ambari.server.api.services.views.ViewUrlsService
class org.apache.ambari.server.api.services.users.UserAuthorizationService
class org.apache.ambari.server.api.rest.KdcServerReachabilityCheck
class org.apache.ambari.server.api.services.ExtensionsService
class org.apache.ambari.server.api.services.SettingService
class org.apache.ambari.server.api.rest.HealthCheck
class org.apache.ambari.server.api.services.AlertTargetService
class org.apache.ambari.server.api.services.RecommendationService
class org.apache.ambari.server.api.services.TargetClusterService
class org.apache.ambari.server.api.services.KerberosDescriptorService
class org.apache.ambari.server.api.services.StacksService
class org.apache.ambari.server.api.services.users.UserService
class org.apache.ambari.server.api.services.KeyService
class org.apache.ambari.server.api.services.BlueprintService
class org.apache.ambari.server.api.services.LdapSyncEventService
class org.apache.ambari.server.api.services.views.ViewPermissionService
class org.apache.ambari.server.api.services.views.ViewDataMigrationService
class org.apache.ambari.server.api.services.RoleAuthorizationService
class org.apache.ambari.server.api.services.groups.MemberService
class org.apache.ambari.server.api.services.groups.GroupPrivilegeService
class org.apache.ambari.server.api.rest.BootStrapResource
class org.apache.ambari.server.api.services.ActionService
class org.apache.ambari.server.api.services.views.ViewInstanceService
class org.apache.ambari.server.api.services.users.UserPrivilegeService
class org.apache.ambari.server.api.services.views.ViewVersionService
class org.apache.ambari.server.api.services.ExtensionLinksService
class org.apache.ambari.server.api.services.PersistKeyValueService
class org.apache.ambari.server.api.services.RequestService
class org.apache.ambari.server.api.services.PermissionService
2020-08-10 06:15:55,284 INFO [main] ScanningResourceConfig:153 - Provider classes found:
class org.apache.ambari.server.api.GsonJsonProvider
2020-08-10 06:15:55,329 INFO [main] WebApplicationImpl:815 - Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
2020-08-10 06:15:55,983 WARN [main] Errors:173 - The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.InstanceService.getInstances(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.InstanceService.getInstance(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.views.ViewService.getViews(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.views.ViewService.getView(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.views.ViewVersionService.getVersions(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.views.ViewVersionService.getVersion(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.RequestService.getRequests(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.RequestService.getRequest(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.BlueprintService.getBlueprints(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.BlueprintService.getBlueprint(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.users.UserService.getUsers(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.users.UserService.getUser(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.HostService.getHosts(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.HostService.getHost(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.KerberosDescriptorService.getKerberosDescriptors(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity.

 

 

2) Postgress DB is fine 

 

Postgress DB and Ambari server running on same host

 

# grep -i jdbc /etc/ambari-server/conf/ambari.properties
server.jdbc.connection-pool=internal
server.jdbc.database=postgres
server.jdbc.database_name=ambari
server.jdbc.postgres.schema=ambari
server.jdbc.user.name=ambari
server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat

 

/etc/ambari-server/conf # service postgresql status
● postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2020-08-10 06:13:32 CEST; 20min ago
Process: 26813 ExecStop=/usr/lib/postgresql-init stop (code=exited, status=0/SUCCESS)
Process: 26783 ExecReload=/usr/lib/postgresql-init reload (code=exited, status=0/SUCCESS)
Process: 26826 ExecStart=/usr/lib/postgresql-init start (code=exited, status=0/SUCCESS)
Main PID: 26838 (postgres)
Tasks: 15 (limit: 512)
CGroup: /system.slice/postgresql.service
├─26838 /usr/lib/postgresql94/bin/postgres -D /var/lib/pgsql/data
├─26839 postgres: logger process
├─26841 postgres: checkpointer process
├─26842 postgres: writer process
├─26843 postgres: wal writer process
├─26844 postgres: autovacuum launcher process
├─26845 postgres: stats collector process
├─26966 postgres: root citrix-confdb [local] idle
├─27110 postgres: ambari ambari 127.0.0.1(44408) idle
├─27111 postgres: ambari ambari 127.0.0.1(44410) idle
├─27130 postgres: ambari ambari 127.0.0.1(44436) idle
├─27204 postgres: ctxvda citrix-confdb 127.0.0.1(44456) idle
├─27253 postgres: ambari ambari 127.0.0.1(44476) idle
├─27254 postgres: ambari ambari 127.0.0.1(44478) idle
└─27255 postgres: ambari ambari 127.0.0.1(44480) idle

Aug 10 06:13:31  systemd[1]: Starting PostgreSQL database server...
Aug 10 06:13:31  postgresql-init[26826]: 2020-08-10 06:13:31 CEST LOG: redirecting log output to logging collector process
Aug 10 06:13:31  postgresql-init[26826]: 2020-08-10 06:13:31 CEST HINT: Future log output will appear in directory "pg_log".
Aug 10 06:13:32  systemd[1]: Started PostgreSQL database server.

 

 

3) Please note postgress and ambari server are running on same host 

 

 

Please let me know your findings

 

Thanks

Contributor

Your repository server isnt accepting your request. The HTTP 403 means forbidden and that is the main problem here. If you have access to the repo server , please check the URL is correct. Also double check that the user you are running has sufficient privileges. If you do not have access, you need to contact your administrator.

New Contributor

hi, could it have any other solutions to solve 403 error ?  like this, I only use local hadoop.tar.gz to initial my cluster. I do not use repository. Could you please give me some suggestions ? thanks a lot.

; ;