question Re: Error while installing component on a host via Ambari REST API in Archives of Support Questions (Read Only)

Error while installing component on a host via Ambari REST API

wgonzalez — Sat, 26 Sep 2015 00:34:01 GMT

I am trying to install a component on a host via API:

$ curl -u admin -i -H "X-Requested-By:ambari" -X PUT \
  -d '{"HostRoles": {"state": "INSTALLED"}}' \
  http://<ambari>:8080/api/v1/clusters/clustername/hosts/node1.hortonworks.com/host_components/ZOOKEEPER

... getting error:

Caused by: java.lang.NullPointerException
at org.apache.ambari.server.state.ConfigHelper.getPropertyValuesWithPropertyType(ConfigHelper.java:489)
at org.apache.ambari.server.controller.AmbariManagementControllerImpl.createHostAction(AmbariManagementControllerImpl.java:1934)
at org.apache.ambari.server.controller.AmbariManagementControllerImpl.doStageCreation(AmbariManagementControllerImpl.java:2336)
at org.apache.ambari.server.controller.AmbariManagementControllerImpl.addStages(AmbariManagementControllerImpl.java:2593)
at org.apache.ambari.server.controller.internal.HostComponentResourceProvider.updateHostComponents(HostComponentResourceProvider.java:612)
at org.apache.ambari.server.controller.internal.HostComponentResourceProvider$4.invoke(HostComponentResourceProvider.java:753)

The operation fails with "Server Error"

Any ideas?

Re: Error while installing component on a host via Ambari REST API

abajwa — Sat, 26 Sep 2015 00:37:11 GMT

@wgonzalez@hortonworks.com there are usually a number of steps to follow when trying to add a service via API. Have you already run the initial steps? See wiki for more details:

https://cwiki.apache.org/confluence/display/AMBARI/Adding+a+New+Service+to+an+Existing+Cluster

Re: Error while installing component on a host via Ambari REST API

wgonzalez — Sat, 26 Sep 2015 00:41:47 GMT

Yes. This is on step 6 of that page. I already did 1-5.

Re: Error while installing component on a host via Ambari REST API

dstreev — Sat, 26 Sep 2015 01:02:11 GMT

Have you tried to create the component first, before attempting the installation?

IE:

curl -i -X -H "X-Requested-By: ambari" POST -u admin:admin http://<ambari-host>:8080/api/v1/clusters/<clustername>/hosts/<host_name>/host_components/ZOOKEEPER.

Re: Error while installing component on a host via Ambari REST API

wgonzalez — Sat, 26 Sep 2015 01:50:18 GMT

Yes Sir. According to the aforementioned website, these are the steps I used:

Step 1 - Add a Service to the cluster

curl -u admin:admin -i -H "X-Requested-By: ambari" -X POST -d '{"ServiceInfo":{"service_name":"ZOOKEEPER"}}' http://127.0.0.1:8080/api/v1/clusters/Sandbox/services

Step 2 - Add Components to the service

curl -u admin:admin -i -H "X-Requested-By: ambari" -X POST http://127.0.0.1:8080/api/v1/clusters/Sandbox/services/ZOOKEEPER/components/ZOOKEEPER_SERVER
curl -u admin:admin -i -H "X-Requested-By: ambari" -X POST http://127.0.0.1:8080/api/v1/clusters/Sandbox/services/ZOOKEEPER/components/ZOOKEEPER_CLIENT

Step 3 & 4 - Create and apply configuration

curl -u admin:admin -i -H "X-Requested-By: ambari" -X PUT -d '{"Clusters" : {"desired_configs": {"type": "zoo.cfg","tag": "20150924","properties": {"autopurge.purgeInterval" : "24","autopurge.snapRetainCount" : "30",          "clientPort" : "2181","dataDir" : "/hadoop/zookeeper","initLimit" : "10","syncLimit" : "5","tickTime" : "2000"}}}}'  http://127.0.0.1:8080/api/v1/clusters/Sandbox 

curl -u admin:admin -i -H "X-Requested-By: ambari" -X PUT -d '{"Clusters" : {"desired_configs": {"type": "zookeeper-env","tag": "20150924","properties": {"content" : "\nexport JAVA_HOME={{java64_home}}\nexport ZOOKEEPER_HOME={{zk_home}}\nexport ZOO_LOG_DIR={{zk_log_dir}}\nexport ZOOPIDFILE={{zk_pid_file}}\nexport SERVER_JVMFLAGS={{zk_server_heapsize}}\nexport JAVA=$JAVA_HOME/bin/java\nexport CLASSPATH=$CLASSPATH:/usr/share/zookeeper/*\n\n{% if security_enabled %}\nexport SERVER_JVMFLAGS=\"$SERVER_JVMFLAGS -Djava.security.auth.login.config={{zk_server_jaas_file}}\"\nexport CLIENT_JVMFLAGS=\"$CLIENT_JVMFLAGS -Djava.security.auth.login.config={{zk_client_jaas_file}}\"\n{% endif %}", "zk_log_dir" : "/var/log/zookeeper","zk_pid_dir" : "/var/run/zookeeper","zk_user" : "zookeeper"}}}}'  http://127.0.0.1:8080/api/v1/clusters/Sandbox 

curl -u admin:admin -i -H "X-Requested-By: ambari" -X PUT -d '{"Clusters" : {"desired_configs": {"type": "zookeeper-log4j","tag": "20150924","properties": {"content" : "\n#\n#\n# Licensed to the Apache Software Foundation (ASF) under one\n# or more contributor license agreements.  See the NOTICE file\n# distributed with this work for additional information\n# regarding copyright ownership.  The ASF licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may not use this file except in compliance\n# with the License.  You may obtain a copy of the License at\n#\n#   http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied.  See the License for the\n# specific language governing permissions and limitations\n# under the License.\n#\n#\n#\n\n#\n# ZooKeeper Logging Configuration\n#\n\n# DEFAULT: console appender only\nlog4j.rootLogger=INFO, CONSOLE\n\n# Example with rolling log file\n#log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE\n\n# Example with rolling log file and tracing\n#log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE\n\n#\n# Log INFO level and above messages to the console\n#\nlog4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender\nlog4j.appender.CONSOLE.Threshold=INFO\nlog4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout\nlog4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n\n\n#\n# Add ROLLINGFILE to rootLogger to get log file output\n#    Log DEBUG level and above messages to a log file\nlog4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender\nlog4j.appender.ROLLINGFILE.Threshold=DEBUG\nlog4j.appender.ROLLINGFILE.File=zookeeper.log\n\n# Max log file size of 10MB\nlog4j.appender.ROLLINGFILE.MaxFileSize=10MB\n# uncomment the next line to limit number of backup files\n#log4j.appender.ROLLINGFILE.MaxBackupIndex=10\n\nlog4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout\nlog4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n\n\n\n#\n# Add TRACEFILE to rootLogger to get log file output\n#    Log DEBUG level and above messages to a log file\nlog4j.appender.TRACEFILE=org.apache.log4j.FileAppender\nlog4j.appender.TRACEFILE.Threshold=TRACE\nlog4j.appender.TRACEFILE.File=zookeeper_trace.log\n\nlog4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout\n### Notice we are including log4js NDC here (%x)\nlog4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n"}}}}'  http://127.0.0.1:8080/api/v1/clusters/Sandbox

Step 5 - Create host components

curl -u admin:admin -i -H "X-Requested-By: ambari" -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"ZOOKEEPER_SERVER"}}] }' http://127.0.0.1:8080/api/v1/clusters/Sandbox/hosts?Hosts/host_name=sandbox.hortonworks.com

curl -u admin:admin -i -H "X-Requested-By: ambari" -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"ZOOKEEPER_CLIENT"}}] }' http://127.0.0.1:8080/api/v1/clusters/Sandbox/hosts?Hosts/host_name=sandbox.hortonworks.com

Step 6 - Install & Start the service

curl -u admin:admin -i -H "X-Requested-By: ambari" -X PUT -d '{"ServiceInfo": {"state" : "INSTALLED"}}' http://127.0.0.1:8080/api/v1/clusters/Sandbox/services/ZOOKEEPER 

curl -u admin:admin -i -H "X-Requested-By: ambari" -X PUT -d '{"ServiceInfo": {"state" : "STARTED"}}' http://127.0.0.1:8080/api/v1/clusters/Sandbox/services/ZOOKEEPER

** INSTALLED is where this is failing

Thanks

Re: Error while installing component on a host via Ambari REST API

wgonzalez — Mon, 28 Sep 2015 22:20:38 GMT

Please let me know if anyone has any ideas regarding this.

Re: Error while installing component on a host via Ambari REST API

dgrinenko — Tue, 29 Sep 2015 06:42:49 GMT

according to the place, where error appears - the problem is with the configs. Couldn't say exactly, some can be missing or wrong

Re: Error while installing component on a host via Ambari REST API

yusaku — Tue, 29 Sep 2015 08:56:56 GMT

1. Why not use Blueprint? Or are you trying to add a service to an existing cluster?

2. Do you have cluster-env with smokeuser defined? It looks like ZooKeeper install fails without it.

curl -u admin:admin -i -H "X-Requested-By: ambari -X PUT -d '{"Clusters" : {"desired_configs": {"type": "cluster-env","tag": "20150924","properties": {"smokeuser":"ambari-qa"}}}}' http://127.0.0.1:8080/api/v1/clusters/Sandbox

should get you thru, but you might want to look at an existing working cluster and define all the standard properties that come in cluster-env.

Re: Error while installing component on a host via Ambari REST API

wgonzalez — Wed, 30 Sep 2015 01:03:35 GMT

Customer does not want Blueprint.

I will check regarding smoke user.

Good tips -- thanks

Re: Error while installing component on a host via Ambari REST API

jspeidel — Fri, 02 Oct 2015 23:33:56 GMT

I am really interested in hearing why the customer doesn't want to use blueprints. Using the low level api's directly to provision a cluster is very hard to get right, that is one of the primary reasons that the higher level blueprint api was introduced.

Re: Error while installing component on a host via Ambari REST API

yusaku — Sat, 03 Oct 2015 00:52:57 GMT

Did adding smokeuser work, @wgonzalez@hortonworks.com?

Re: Error while installing component on a host via Ambari REST API

nsomasundaram — Fri, 23 Oct 2015 00:31:41 GMT

Are you getting an error when trying to install a component or service?

$ curl -u admin -i -H "X-Requested-By:ambari"-X PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://<ambari>:8080/api/v1/clusters/clustername/hosts/node1.hortonworks.com/host_components/ZOOKEEPER

The above API will fail because ZOOKEEPER is not the name of a host component. It should either be ZOOKEEPER_CLIENT or ZOOKEEPEER_SERVER. On the trunk build I get a NoSuchResourceException when using ZOOKEEPER as the component name. With the correct component name, the API succeeded.

The API to install the ZOOKEEPER service was successful:

curl -u admin:admin -i -H "X-Requested-By: ambari"-X PUT -d '{"ServiceInfo": {"state" : "INSTALLED"}}' http://127.0.0.1:8080/api/v1/clusters/Sandbox/services/ZOOKEEPER