Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Server Error installing Storm through ambari

Labels:
theyaa
Explorer

Created ‎03-08-2017 03:50 PM

Trying to install Storm through ambari, everytime I reach step 6, the deployment stage I get hit with a Server error and never installs the services. Ambari server is complaining about storm-env.xml. Below is the content of storm-env.xml and the exception I see in ambari-server. I would appreciate your suggestions.

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->

<configuration>
  <property>
    <name>storm_user</name>
    <display-name>Storm User</display-name>
    <value>storm</value>
    <property-type>USER</property-type>
    <description/>
  </property>
  <property>
    <name>storm_log_dir</name>
    <value>/var/log/storm</value>
    <description/>
  </property>
  <property>
    <name>storm_pid_dir</name>
    <value>/var/run/storm</value>
    <description/>
  </property>
  <property>
    <name>jmxremote_port</name>
    <value>56431</value>
    <description/>
  </property>
  <property>
    <name>storm_principal_name</name>
    <description>Storm principal name</description>
  </property>
  <property>
    <name>storm_principal_name</name>
    <description>Storm principal name</description>
  </property>
  <property>
    <name>storm_keytab</name>
    <description>Storm keytab path</description>
  </property>
  <property>
    <name>storm_ui_principal_name</name>
    <description>Storm UI principal name</description>
  </property>
  <property>
    <name>storm_ui_keytab</name>
    <description>Storm UI keytab path</description>
  </property>
  <property>
    <name>nimbus_keytab</name>
    <description>Nimbus keytab path</description>
  </property>
  <property>
    <name>nimbus_principal_name</name>
    <description>Nimbus principal name</description>
  </property>
  <property>
    <name>storm_user_nofile_limit</name>
    <value>128000</value>
    <description>Max open files limit setting for STORM user.</description>
  </property>
  <property>
    <name>storm_user_nproc_limit</name>
    <value>65536</value>
    <description>Max number of processes limit setting for STORM user.</description>
  </property>
  <!-- storm-env.sh -->
  <property>
    <name>content</name>
    <display-name>storm-env template</display-name>
    <description>This is the jinja template for storm-env.sh file</description>
    <value>
#!/bin/bash

# Set Storm specific environment variables here.

# The java implementation to use.
export JAVA_HOME={{java64_home}}
      export STORM_CONF_DIR={{conf_dir}}
      export STORM_HOME={{storm_component_home_dir}}

      export STORM_JAR_JVM_OPTS={{jar_jvm_opts}}
    </value>
  </property>
  <property>
    <name>nimbus_seeds_supported</name>
    <value>true</value>
    <description/>
  </property>
  <property>
    <name>storm_logs_supported</name>
    <value>true</value>
    <description/>
  </property>
</configuration>

08 Mar 2017 02:10:19,998  INFO [qtp-client-19] ClusterImpl:2094 - Looking for service for config types [storm-env]

08 Mar 2017 02:10:19,998  INFO [qtp-client-19] ClusterImpl:2110 - Service STORM returning

08 Mar 2017 02:10:20,001  WARN [qtp-client-19] ServletHandler:563 - /api/v1/clusters/BI4_QSE

java.lang.NullPointerException

	at org.apache.ambari.server.controller.AmbariManagementControllerImpl.replaceMaskedPasswords(AmbariManagementControllerImpl.java:799)

	at org.apache.ambari.server.controller.AmbariManagementControllerImpl.createConfig(AmbariManagementControllerImpl.java:780)

	at org.apache.ambari.server.controller.AmbariManagementControllerImpl.createConfiguration(AmbariManagementControllerImpl.java:724)

	at org.apache.ambari.server.controller.AmbariManagementControllerImpl.updateCluster(AmbariManagementControllerImpl.java:1398)

	at org.apache.ambari.server.controller.AmbariManagementControllerImpl.updateClusters(AmbariManagementControllerImpl.java:1286)

	at org.apache.ambari.server.controller.internal.ClusterResourceProvider$2.invoke(ClusterResourceProvider.java:241)

	at org.apache.ambari.server.controller.internal.ClusterResourceProvider$2.invoke(ClusterResourceProvider.java:238)

	at org.apache.ambari.server.controller.internal.AbstractResourceProvider.modifyResources(AbstractResourceProvider.java:331)

	at org.apache.ambari.server.controller.internal.ClusterResourceProvider.updateResources(ClusterResourceProvider.java:238)

	at org.apache.ambari.server.controller.internal.ClusterControllerImpl.updateResources(ClusterControllerImpl.java:310)

	at org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.update(PersistenceManagerImpl.java:104)

	at org.apache.ambari.server.api.handlers.UpdateHandler.persist(UpdateHandler.java:42)

	at org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:72)

	at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:135)

	at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:105)

	at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:74)

	at org.apache.ambari.server.api.services.ClusterService.updateCluster(ClusterService.java:151)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:497)

	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)

	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)

	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)

	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)

	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)

	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)

	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)

	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)

	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)

	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)

	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)

	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540)

	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715)

	at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)

	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)

	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)

	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:182)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)

	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)

	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)

	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)

	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)

	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)

	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)

	at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)

	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)

	at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)

	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)

	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)

	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)

	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)

	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)

	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)

	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)

	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)

	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)

	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)

	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)

	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)

	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)

	at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:209)

	at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:198)

	at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:132)

	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)

	at org.eclipse.jetty.server.Server.handle(Server.java:370)

	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)

	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)

	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)

	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)

	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)

	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)

	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)

	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)

	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)

	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)

	at java.lang.Thread.run(Thread.java:745)
588 Views
0 Kudos
4 REPLIES 4

jsensharma
Super Mentor

Created ‎03-08-2017 04:51 PM

@Theyaa Matti

It will be helpful to know which version of ambari is it ? Because i am not able to find the "replaceMaskedPasswords" method in the latest ambari code in that place.

org.apache.ambari.server.controller.AmbariManagementControllerImpl.replaceMaskedPasswords

- Also can you please check the ambari-server.log again and findout if you are getting many NullPointerException immediately the following line? Or above is the only place.

WARN [qtp-client-19] ServletHandler:563 - /api/v1/clusters/BI4_QSE 
java.lang.NullPointerException

.

388 Views
0 Kudos

theyaa
Explorer

Created ‎03-08-2017 05:06 PM

The ambari version is 2.1 and I only have this exception. I am wondering if there is a way to see if a property in that storm-env.xml is causing the issue. Maybe there is a property attribute or something. Is there a way to find that? or a way to validate the config against ambari?

388 Views
0 Kudos

jsensharma
Super Mentor

Created ‎03-08-2017 05:14 PM

@Theyaa Matti

If we will know the exact version of ambari (even the minor version) then we can match the line number which is causing the NullPointerException to get a hint that what is becoming null there.

ambari-server  --version

- But unfortunately when i am checking the exact same code for ambari 2.1 version then i do not see any such method "replaceMaskedPasswords" there.

https://github.com/apache/ambari/blob/release-2.1.0/ambari-server/src/main/java/org/apache/ambari/se...

- But one thing i am suspecting that might be the case. Have you by any chance used the following command recently to encrypt all the passwords in ambari? If yes, then it is possible that some of the password is not encrypted properly or might be null.

ambari-server encrypt-passwords

.

388 Views
0 Kudos

theyaa
Explorer

Created ‎03-08-2017 08:31 PM

The ambari version is 2.1.0 and I never executed the command ambari-server encrypt-passwords

I managed to get the services installed but everytime I try to start drpc or nimbus I get the following error:

2017-03-08 05:19:45.847 main o.a.s.d.nimbus [INFO] Starting nimbus server for storm version '1.0.3'

2017-03-08 05:19:45.848 main o.a.s.s.a.ThriftServer [ERROR] ThriftServer is being stopped due to: java.lang.RuntimeException: java.lang.ClassNotFoundException:

java.lang.RuntimeException: java.lang.ClassNotFoundException:

        at org.apache.storm.security.auth.AuthUtils.GetTransportPlugin(AuthUtils.java:267)

        at org.apache.storm.security.auth.ThriftServer.serve(ThriftServer.java:65)

        at org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2212)

        at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2236)

        at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2259)

        at clojure.lang.AFn.applyToHelper(AFn.java:152)

        at clojure.lang.AFn.applyTo(AFn.java:144)

        at org.apache.storm.daemon.nimbus.main(Unknown Source)

Caused by: java.lang.ClassNotFoundException:

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:264)

        at org.apache.storm.security.auth.AuthUtils.GetTransportPlugin(AuthUtils.java:263)

        ... 7 more

I added the storm lib dir to the java.library.path

and I changed the value of nimbus.topology.validator to be "org.apache.storm.nimbus.DefaultTopologyValidator"

Not sure what the issue is that it still can't find that file.

388 Views
0 Kudos
Don't have an account?
Register
Announcements
What's New @ Cloudera
Cloudera Streaming Analytics 7.2.15 for Public Cloud - New Features
What's New @ Cloudera
Cloudera Streams Messaging 7.2.15 for Public Cloud - New Features
What's New @ Cloudera
Cloudera Streaming Analytics 1.7.0 For Private Cloud - New Features
What's New @ Cloudera
Cloudera DataFlow for the Public Cloud introduces Inbound Connections and supports Apache NiFi 1.16
Product Announcements
CDP Public Cloud Release Summary: April 2022
View More Announcements