Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Trouble with installing HDP 2.6.4 through Ambari using local repo

avatar

I am trying to install HDP 2.6.4 version through Ambari 2.6.1 - the Ambari GUI is up and running.

Created the local repo in another server. Able to get to the local repos from the browser using the Base URLs.

In Ambari Cluster install wizard, selected following as local repos:

HDP-2.6 - http://1.1.1.1/repo/HDP/centos7/2.x/updates/2.6.4.0/

HDP-2.6-GPL -http://1.1.1.1/repo/HDP-GPL/centos7/2.6.4.0-91

HDP-UTILS-1.1.0.22 - http://1.1.1.1/repo/HDP-UTILS/repos/centos7/

But with this, the Ambari throws Server Error (500).

I have tried adding the hostnames in /etc/hosts and tried with FQDN names as well.

I am a beginner and any help is appreciated. Thanks!

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Jay

Sometimes it happens that the "wget" works because the host has some global proxy settings defined (like in ~/.bash_profile, /etc/profile, /etc/wgetrc file , However ambari is a java process and it requires oproxy settings done via JVM properties. As described in https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.0/bk_ambari-administration/content/ch_setting...

-Dhttp.proxyHost=<yourProxyHost> -Dhttp.proxyPort=<yourProxyPort>

So can you please check your ambari server host if there is any Http Proxy setting done at global level? If yes then we will need to add the proxy setting in "/var/lib/ambari-server/ambari-env.sh" file as well.

View solution in original post

10 REPLIES 10

avatar
Master Mentor

@Jay

Whenever there is a 500 error you must see some long stacktrace in the ambari-server.og.

So can you please share the complete error that you are receiving in the "/var/log/ambari-server/ambari-server.log" ?

Also, Please let us know how you are updating the 2.6.4.0 local repo to ambari using the VDF (Version definition file) ? If yes then please share the VDF file as well.

avatar

@Jay Kumar SenSharma

I downloaded the VDF from http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/HDP-2.6.4.0-91.xml and used it in Ambari.

This is the error from /var/log/ambari-server/ambari-server.log

ERROR [ambari-client-thread-2234] ContainerResponse:419 - The RuntimeException could not be mapped to a response, re-throwing to the HTTP container

java.lang.RuntimeException: java.lang.NullPointerException

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1488)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)

at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3018)

at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:489)

at org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)

at org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)

at org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)

at org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)

at org.apache.ambari.server.controller.AmbariManagementControllerImpl.verifyRepository(AmbariManagementControllerImpl.java:4628)

at org.apache.ambari.server.controller.AmbariManagementControllerImpl.updateRepositories(AmbariManagementControllerImpl.java:4562)

at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$4.invoke(RepositoryResourceProvider.java:140)

at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$4.invoke(RepositoryResourceProvider.java:137)

at org.apache.ambari.server.controller.internal.AbstractResourceProvider.invokeWithRetry(AbstractResourceProvider.java:455)

at org.apache.ambari.server.controller.internal.AbstractResourceProvider.modifyResources(AbstractResourceProvider.java:336)

at org.apache.ambari.server.controller.internal.RepositoryResourceProvider.updateResources(RepositoryResourceProvider.java:137)

at org.apache.ambari.server.controller.internal.ClusterControllerImpl.updateResources(ClusterControllerImpl.java:319)

at org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.update(PersistenceManagerImpl.java:125)

at org.apache.ambari.server.api.handlers.UpdateHandler.persist(UpdateHandler.java:45)

at org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:73)

at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)

at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:126)

at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:90)

at org.apache.ambari.server.api.services.RepositoryService.updateRepository(RepositoryService.java:145)

at sun.reflect.GeneratedMethodAccessor151.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)

at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)

at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)

at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)

at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:287)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:132)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)

at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)

at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)

at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)

at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)

at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)

at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)

at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)

at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)

at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212)

at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201)

at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:139)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)

at org.eclipse.jetty.server.Server.handle(Server.java:370)

at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)

at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:984)

at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1045)

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)

at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236)

at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)

at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.NullPointerException

at com.sun.security.ntlm.Client.type3(Client.java:161)

at sun.net.www.protocol.http.ntlm.NTLMAuthentication.buildType3Msg(NTLMAuthentication.java:241)

at sun.net.www.protocol.http.ntlm.NTLMAuthentication.setHeaders(NTLMAuthentication.java:216)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1643)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)

at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)

... 105 more

01 Feb 2018 11:35:53,591WARN [ambari-client-thread-2234] ServletHandler:561 - Error Processing URI: /api/v1/stacks/HDP/versions/2.6/operating_systems/redhat7/repositories/HDP-UTILS-1.1.0.22 - (java.lang.RuntimeException) java.lang.NullPointerException

01 Feb 2018 11:35:53,591WARN [ambari-client-thread-2234] ServletHandler:561 - Error Processing URI: /api/v1/stacks/HDP/versions/2.6/operating_systems/redhat7/repositories/HDP-UTILS-1.1.0.22 - (java.lang.RuntimeException) java.lang.NullPointerException

avatar

@Jay Kumar SenSharma

Same error message for HDP and HDP-GPL as well.

avatar
Master Mentor

@Jay

The NullPointerException indicates that the "spec" (URL location) is not setup correctly. Hence it is not able to find the Repo which is causing this error.

01 Feb 2018 11:35:53,591WARN [ambari-client-thread-2234] ServletHandler:561 - Error Processing URI: 
/api/v1/stacks/HDP/versions/2.6/operating_systems/redhat7/repositories/HDP-UTILS-1.1.0.22  - (java.lang.RuntimeException) java.lang.NullPointerException

.

So can you please try enabling the DEBUG on the ambari-serevr by editing the "/etc/ambari-server/conf/log4j.properties" file and edit the INFO to DEBUG in the following line:

# grep 'log4j.rootLogger' /etc/ambari-server/conf/log4j.properties
log4j.rootLogger=INFO,file

.

Then restart the ambari-server and then once you get the same error please grep the string "readFrom spec" in the log and then share it with us.

# grep 'readFrom spec' /var/log/ambari-server/ambari-server.log

.

Then from ambari-server Host can you try doing a "wget" to the repo to see if you are able to do it? This will isolate any Firewall or Proxy Server issue.

# wget http://1.1.1.1/repo/HDP-UTILS/repos/centos7/repodata/repomd.xml

Reference: https://github.com/apache/ambari/blob/release-2.6.1/ambari-server/src/main/java/org/apache/ambari/se...

avatar

@Jay Kumar SenSharma

Thanks for the quick reply. Here are the details:

01 Feb 2018 14:35:57,714 DEBUG [Stack Version Loading Thread] URLStreamProvider:178 - readFrom spec:http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json

01 Feb 2018 14:37:01,602 DEBUG [ambari-client-thread-81] URLStreamProvider:178 - readFrom spec:http://1.1.1.1/repo/HDP/centos7/2.x/updates/2.6.4.0/repodata/repomd.xml

01 Feb 2018 14:37:01,612 DEBUG [ambari-client-thread-40] URLStreamProvider:178 - readFrom spec:http://1.1.1.1/repo/HDP-GPL/centos7/2.6.4.0-91/repodata/repomd.xml

01 Feb 2018 14:37:01,636 DEBUG [ambari-client-thread-41] URLStreamProvider:178 - readFrom spec:http://1.1.1.1/repo/HDP-UTILS/repos/centos7/repodata/repomd.xml

avatar
Master Mentor

@Jay

As we see the URL as following so please try to do a WGET from ambari server host to that url to see if it si working or not?

# wget http://1.1.1.1/repo/HDP/centos7/2.x/updates/2.6.4.0/repodata/repomd.xml
# wget http://1.1.1.1/repo/HDP-UTILS/repos/centos7/repodata/repomd.xml
# wget http://1.1.1.1/repo/HDP-GPL/centos7/2.6.4.0-91/repodata/repomd.xml

.

This will isolate any Firewall or Proxy Server issue. And if there is any issue with the Local repo configuration then we might not be able to do wget .. it might fail.

avatar

@Jay Kumar SenSharma

Was able to wget successfully for these 3 XMLs.

Not sure if the json matters, but that didnt work - http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json

avatar
Master Mentor

@Jay

Sometimes it happens that the "wget" works because the host has some global proxy settings defined (like in ~/.bash_profile, /etc/profile, /etc/wgetrc file , However ambari is a java process and it requires oproxy settings done via JVM properties. As described in https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.0/bk_ambari-administration/content/ch_setting...

-Dhttp.proxyHost=<yourProxyHost> -Dhttp.proxyPort=<yourProxyPort>

So can you please check your ambari server host if there is any Http Proxy setting done at global level? If yes then we will need to add the proxy setting in "/var/lib/ambari-server/ambari-env.sh" file as well.

avatar

@Jay Kumar SenSharma

I removed some proxy settings from /var/lib/ambari-server/ambari-env.sh and it worked 🙂

Thank you for your time and help!