Created on 05-05-2017 05:58 PM - edited 09-16-2022 04:34 AM
Hi guys,
I've been trying to install SAM TP2 on an existing 2.6 cluster I have (I'll need to replicate the same on a customer's POC env)
Getting the following error on Install of NiFi
2017-05-05 17:09:31,122 - Could not determine stack version for component nifi by calling '/usr/bin/hdp-select status nifi > /tmp/tmpEy4hbD'. Return Code: 1, Output: ERROR: Invalid package - nifi Packages: accumulo-client accumulo-gc accumulo-master accumulo-monitor accumulo-tablet accumulo-tracer atlas-client atlas-server beacon beacon-client beacon-server druid-broker druid-coordinator druid-historical druid-middlemanager druid-overlord druid-router druid-superset falcon-client falcon-server flume-server hadoop-client hadoop-hdfs-datanode hadoop-hdfs-journalnode hadoop-hdfs-namenode hadoop-hdfs-nfs3 hadoop-hdfs-portmap hadoop-hdfs-secondarynamenode hadoop-hdfs-zkfc hadoop-httpfs hadoop-mapreduce-historyserver hadoop-yarn-nodemanager hadoop-yarn-resourcemanager hadoop-yarn-timelineserver hbase-client hbase-master hbase-regionserver hive-metastore hive-server2 hive-server2-hive2 hive-webhcat kafka-broker knox-server livy-server livy2-server mahout-client oozie-client oozie-server phoenix-client phoenix-server ranger-admin ranger-kms ranger-tagsync ranger-usersync slider-client spark-client spark-historyserver spark-thriftserver spark2-client spark2-historyserver spark2-thriftserver sqoop-client sqoop-server storm-client storm-nimbus storm-slider-client storm-supervisor zeppelin-server zookeeper-client zookeeper-server Aliases: accumulo-server all client hadoop-hdfs-server hadoop-mapreduce-server hadoop-yarn-server hive-server . Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 304, in <module> Master().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 313, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 67, in install self.configure(env, True) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 116, in locking_configure original_configure(obj, *args, **kw) File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 135, in configure File(format("{params.bin_dir}/nifi-env.sh"), content=env_content, owner=params.nifi_user, group=params.nifi_group, mode=0755) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 120, in action_create raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname)) resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/nifi/bin/nifi-env.sh'] failed, parent directory /usr/hdp/current/nifi/bin doesn't exist
I think this is the relevant bit:
resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/nifi/bin/nifi-env.sh'] failed, parent directory /usr/hdp/current/nifi/bin doesn't exist
Then if i create the symbolic link as follows:
ln -s /usr/hdf/3.0.0.0-264/nifi/ /usr/hdp/current/nifi
I get:
resource_management.core.exceptions.Fail: Applying Directory['/usr/hdp/current/nifi/conf'] failed, looped symbolic links found while resolving /usr/hdp/current/nifi/conf
So clearly that's not right... I must have gone wrong somewhere, but where? Interestingly Registry and Streaming Analytics Manager itself install fine??? odd
Thanks in advance!
Created 05-05-2017 06:13 PM
FINAL EDIT: Turns out this is expected because you need HDP 2.6.1 and Ambari 2.5.1 to make this work. Solved!
Fixed it lol. Turns out there was already a nifi dir in current which contains a conf and a lib. So i just had to link the bin??
ln -s /usr/hdf/3.0.0.0-264/nifi/bin /usr/hdp/current/nifi/bin
Then I get this and the install succeeds:
[root@ana-smartsense14test0 current]# ll /usr/hdp/current/nifi/ total 8 lrwxrwxrwx. 1 nifi nifi 29 May 5 18:10 bin -> /usr/hdf/3.0.0.0-264/nifi/bin drwxr-xr-x. 2 nifi nifi 4096 May 5 18:10 conf drwxr-xr-x. 2 nifi nifi 4096 May 5 18:09 lib
Doesn't seem correct - is this a bug or was it pibkac?
EDIT: I thought I fixed it, but Ambari is behaving oddly. NiFi starts and shows green, but no process is actually running on the machine. There's no pid file in /var/run/nifi and no logs are generated for nifi, meaning it never started. Manually running the sh script gives me this error:
[nifi@ana-smartsense14test0 bin]$ ./nifi.sh start Java home: /usr/jdk64/jdk1.8.0_112 NiFi home: /usr/hdp/current/nifi Bootstrap Config File: /usr/hdp/current/nifi/conf/bootstrap.conf Error: Could not find or load main class org.apache.nifi.bootstrap.RunNiFi