Could not determine stack version for component nifi by calling '/usr/bin/hdp-select status nifi

[#1]

Hi guys,

I've been trying to install SAM TP2 on an existing 2.6 cluster I have (I'll need to replicate the same on a customer's POC env)

Getting the following error on Install of NiFi

2017-05-05 17:09:31,122 - Could not determine stack version for component nifi by calling '/usr/bin/hdp-select status nifi > /tmp/tmpEy4hbD'. Return Code: 1, Output: ERROR: Invalid package - nifi

Packages:
  accumulo-client
  accumulo-gc
  accumulo-master
  accumulo-monitor
  accumulo-tablet
  accumulo-tracer
  atlas-client
  atlas-server
  beacon
  beacon-client
  beacon-server
  druid-broker
  druid-coordinator
  druid-historical
  druid-middlemanager
  druid-overlord
  druid-router
  druid-superset
  falcon-client
  falcon-server
  flume-server
  hadoop-client
  hadoop-hdfs-datanode
  hadoop-hdfs-journalnode
  hadoop-hdfs-namenode
  hadoop-hdfs-nfs3
  hadoop-hdfs-portmap
  hadoop-hdfs-secondarynamenode
  hadoop-hdfs-zkfc
  hadoop-httpfs
  hadoop-mapreduce-historyserver
  hadoop-yarn-nodemanager
  hadoop-yarn-resourcemanager
  hadoop-yarn-timelineserver
  hbase-client
  hbase-master
  hbase-regionserver
  hive-metastore
  hive-server2
  hive-server2-hive2
  hive-webhcat
  kafka-broker
  knox-server
  livy-server
  livy2-server
  mahout-client
  oozie-client
  oozie-server
  phoenix-client
  phoenix-server
  ranger-admin
  ranger-kms
  ranger-tagsync
  ranger-usersync
  slider-client
  spark-client
  spark-historyserver
  spark-thriftserver
  spark2-client
  spark2-historyserver
  spark2-thriftserver
  sqoop-client
  sqoop-server
  storm-client
  storm-nimbus
  storm-slider-client
  storm-supervisor
  zeppelin-server
  zookeeper-client
  zookeeper-server
Aliases:
  accumulo-server
  all
  client
  hadoop-hdfs-server
  hadoop-mapreduce-server
  hadoop-yarn-server
  hive-server
.
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 304, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 313, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 67, in install
    self.configure(env, True)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 116, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 135, in configure
    File(format("{params.bin_dir}/nifi-env.sh"), content=env_content, owner=params.nifi_user, group=params.nifi_group, mode=0755) 
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 120, in action_create
    raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/nifi/bin/nifi-env.sh'] failed, parent directory /usr/hdp/current/nifi/bin doesn't exist

I think this is the relevant bit:

resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/nifi/bin/nifi-env.sh'] failed, parent directory /usr/hdp/current/nifi/bin doesn't exist

Then if i create the symbolic link as follows:

ln -s /usr/hdf/3.0.0.0-264/nifi/ /usr/hdp/current/nifi

I get:

resource_management.core.exceptions.Fail: Applying Directory['/usr/hdp/current/nifi/conf'] failed, looped symbolic links found while resolving /usr/hdp/current/nifi/conf

So clearly that's not right... I must have gone wrong somewhere, but where? Interestingly Registry and Streaming Analytics Manager itself install fine??? odd

Thanks in advance!

[#2]

FINAL EDIT: Turns out this is expected because you need HDP 2.6.1 and Ambari 2.5.1 to make this work. Solved!

Fixed it lol. Turns out there was already a nifi dir in current which contains a conf and a lib. So i just had to link the bin??

ln -s /usr/hdf/3.0.0.0-264/nifi/bin /usr/hdp/current/nifi/bin

Then I get this and the install succeeds:

[root@ana-smartsense14test0 current]# ll /usr/hdp/current/nifi/
total 8
lrwxrwxrwx. 1 nifi nifi   29 May  5 18:10 bin -> /usr/hdf/3.0.0.0-264/nifi/bin
drwxr-xr-x. 2 nifi nifi 4096 May  5 18:10 conf
drwxr-xr-x. 2 nifi nifi 4096 May  5 18:09 lib

Doesn't seem correct - is this a bug or was it pibkac?

EDIT: I thought I fixed it, but Ambari is behaving oddly. NiFi starts and shows green, but no process is actually running on the machine. There's no pid file in /var/run/nifi and no logs are generated for nifi, meaning it never started. Manually running the sh script gives me this error:

[nifi@ana-smartsense14test0 bin]$ ./nifi.sh start


Java home: /usr/jdk64/jdk1.8.0_112
NiFi home: /usr/hdp/current/nifi


Bootstrap Config File: /usr/hdp/current/nifi/conf/bootstrap.conf


Error: Could not find or load main class org.apache.nifi.bootstrap.RunNiFi