Hi all
Help with the steps to add NiFi service to a an existing HDP cluster using the Ambari REST API.
I have used cloudbreak to deploy a HDP cluster and would like to use Ambari API to automate the installation of NIFI services on the cluster.
Cluster details:
Ambari Version: 2.6.1.3
Base Url: http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.3
HDP Version: 2.6
Stack Repo Id: HDP
Stack Repository Version: 2.6.4.0-91
I am able to install NiFi services successfully to the HDP cluster but have errors when I tried to start the services.
Here are my steps:
#install mpack
wget http://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/tars/hdf_ambari_mp/hdf-ambari-m...
ambari-server install-mpack --mpack=hdf-ambari-mpack-3.1.1.0-35.tar.gz --verbose
ambari-server restart
#install Nifi Services
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X POST https://localhost/ambari/api/v1/clusters/test-cluster/services/NIFI
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X POST https://localhost/ambari/api/v1/clusters/test-cluster/services/NIFI/components/NIFI_MASTER
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X POST https://localhost/ambari/api/v1/clusters/test-cluster/hosts/ip-10-4-2-254.eu-west-1.compute.internal...
#as part of the prerequisites I have exported NIFI configurations from an existing HPD cluster. I am using those configs to setup this new NIFI install.
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-ambari-config --file=nifi-ambari-config.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-ambari-ssl-config --file=nifi-ambari-ssl-config.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-authorizers-env --file=nifi-authorizers-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-bootstrap-env --file=nifi-bootstrap-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-bootstrap-notification-services-env --file=nifi-bootstrap-notification-services-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-env --file=nifi-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-flow-env --file=nifi-flow-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-jaas-conf --file=nifi-jaas-conf.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-login-identity-providers-env --file=nifi-login-identity-providers-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-node-logback-env --file=nifi-node-logback-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-properties --file=nifi-properties.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-state-management-env --file=nifi-state-management-env.json
/var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=test-cluster --config-type=nifi-toolkit-env --file=nifi-toolkit-env.json
#Install Nifi
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context" :"Install Nifi"}, "Body": {"ServiceInfo": {"maintenance_state" : "OFF", "state": "INSTALLED"}}}' https://localhost/ambari/api/v1/clusters/test-cluster/services/NIFI
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context" :"Start NIFI"}, "Body": {"ServiceInfo": {"maintenance_state" : "OFF", "state": "STARTED"}}}' https://localhost/ambari/api/v1/clusters/test-cluster/services/NIFI
The install is successful but the last command to start the nifi service gives the following error on Ambari.
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 231, in <module>
Master().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 152, in start
self.configure(env, is_starting = True)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 120, in locking_configure
original_configure(obj, *args, **kw)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 101, in configure
if not sudo.path_isfile(params.nifi_flow_config_dir+'/flow.xml.gz') and nifi_toolkit_util.existing_cluster(params):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi_toolkit_util.py", line 314, in existing_cluster
zk_command_location = os.path.join(params.zk_root, params.zk_stack_version_buildnum, "zookeeper", "bin", "zkCli.sh")
File "/usr/lib64/python2.7/posixpath.py", line 75, in join
if b.startswith('/'):
AttributeError: 'NoneType' object has no attribute 'startswith'
Has anyone else seen this issue, or have a suggestion on how to get this to work?
Many thanks in advance.
TL
