Not able to configure Ranger plugin for Nifi

[#1]

When we are trying to implement Ranger Plugin in Nifi in our ambari clsuter, we are getting the attached error. We have configured Nifi in distributed mode in two different servers.Please find the attached error log.

Ambari Version :- 2.7.3.0

HDP Version :- 3.1.0.0

HDF version :- 3.3.1.0-10

Nifi Version:- 1.8.0.3.3.1.0-10

stderr:
2019-08-22 17:08:51,422 - Stack selector path does not exist
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 304, in <module>
Master().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 144, in start
setup_ranger_nifi(upgrade_type=None)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/setup_ranger_nifi.py", line 85, in setup_ranger_nifi
component_user_keytab=params.ranger_nifi_keytab if params.security_enabled else None, cred_lib_path_override = cred_lib_prefix_path, cred_setup_prefix_override = cred_setup_prefix_path)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/setup_ranger_plugin_xml.py", line 74, in setup_ranger_plugin
stack_version = get_stack_version(component_select_name)
File "/usr/lib/ambari-agent/lib/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/get_stack_version.py", line 84, in get_stack_version
'Unable to determine the current version because of a non-zero return code of {0}'.format(str(return_code)))
resource_management.core.exceptions.Fail: Unable to determine the current version because of a non-zero return code of 1

[#2]

Can anyone help here, please?

[#3]

@Amir_haikh  @kwabstian53 @jsensharma 

Please help here.

[#4]

@pritam_konar

Looks like it is failing even before the nifi is getting started.
So can you please check and share what is the output of the following commands on the failing host?

# hdf-select | grep nifi
# cat /etc/redhat-release

 
Or please let us know what is the OS type or version?    As we see that the error is coming at the execution of 'get_stack_version' so it will be good to know the "hdf-select" output and also output of the following directories/listing:

# ls -lart /usr/hdf
# ls -lart /usr/hdf/current/

[#5]

Nifi is installed in two servers. Please find the output from both of them:-

1) First Server:-

[root@nladfmrvu11 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# ls -lart /usr/hdf
total 4
drwxr-xr-x. 6 root root 66 Jul 29 15:21 3.3.1.0-10
drwxr-xr-x. 18 root root 4096 Aug 13 20:46 ..
drwxr-xr-x. 5 root root 48 Aug 21 11:17 .
drwxr-xr-x. 2 root root 17 Aug 21 11:18 None
drwxr-xr-x. 2 root root 36 Aug 23 15:55 current
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# ls -lart /usr/hdf/current/
total 0
lrwxrwxrwx. 1 root root 24 Jul 30 11:50 nifi -> /usr/hdf/3.3.1.0-10/nifi
lrwxrwxrwx. 1 root root 32 Jul 30 11:52 nifi-toolkit -> /usr/hdf/3.3.1.0-10/nifi-toolkit
drwxr-xr-x. 5 root root 48 Aug 21 11:17 ..
drwxr-xr-x. 2 root root 36 Aug 23 15:55 .
[root@nladfmrvu11 ~]#

2) Second Server:-

[root@nladfmrvu12 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# ls -lart /usr/hdf
total 4
drwxr-xr-x. 17 root root 4096 Aug 1 10:04 ..
drwxr-xr-x. 5 root root 46 Aug 1 10:04 3.3.1.0-10
drwxr-xr-x. 2 root root 36 Aug 1 10:30 current
drwxr-xr-x. 5 root root 48 Aug 23 11:41 .
drwxr-xr-x. 2 root root 17 Aug 23 11:41 None
[root@nladfmrvu12 ~]# ls -lart /usr/hdf/current/
total 0
lrwxrwxrwx. 1 root root 24 Aug 1 10:11 nifi -> /usr/hdf/3.3.1.0-10/nifi
lrwxrwxrwx. 1 root root 32 Aug 1 10:12 nifi-toolkit -> /usr/hdf/3.3.1.0-10/nifi-toolkit
drwxr-xr-x. 2 root root 36 Aug 1 10:30 .
drwxr-xr-x. 5 root root 48 Aug 23 11:41 ..
[root@nladfmrvu12 ~]#

[#6]

@pritam_konar 

How did you install NiFi on those two hosts?  Via Ambari or Manually?    Looks like somehow the "hdf-select" installation is missing on the Nifi Host.

I was expecting to see the output from the following command:

# hdf-select | grep nifi
nifi - 3.3.1.0-10
nifi-registry - 3.3.1.0-10
nifi-toolkit - 3.3.1.0-10

 .

Looks like hdf-select is not installed on your host.

Can you please try this to see if it works?
Install the hdf-select manually as following:

# yum info hdf-select
# yum clean all
# yum install hdf-select

Then set the correct version for the stack.

# hdf-select set nifi 3.3.1.0-10
# hdf-select set nifi-registry 3.3.1.0-10
# hdf-select set nifi-toolkit 3.3.1.0-10

.

Then try to enable ranger . nifi plugin again.

.
If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

[#7]

Thanks @jsensharma for the reply. I just want a confirmation:-

If i run the command :- 

yum clean all

will it remove all the installed yum packages, or it will remove only  the 'hdf-select' packages?

[#8]

@pritam_konar 

"#yum clean all"  will not delete any package from the host. It will just clean the yum cache header/packages.

# yum clean packages    Eliminate any cached packages from the system. Note that packages are not automatically deleted after they are downloaded.

# yum clean headers  Eliminate all of the header files which yum uses for dependency resolution.

# yum clean all      Runs yum clean packages and yum clean headers as above.

[#9]

I thing hdf-select is installed already in both the cluster. Please find the output below and confirm:-

Please note , we installed Nifi via apache ambari.

[root@nladfmrvu11 ~]# yum info hdf-select
Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
Installed Packages
Name : hdf-select
Arch : noarch
Version : 3.3.1.0
Release : 10
Size : 29 k
Repo : installed
From repo : HDF-3.3-repo-152
Summary : hdf-select Distro select package
License : APL2
Description : hdf-select-3.3.1.0 select package

Uploading Enabled Repositories Report
Loaded plugins: langpacks, product-id, subscription-manager
[root@nladfmrvu11 ~]#

[root@nladfmrvu12 ~]# yum info hdf-select
Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
Installed Packages
Name : hdf-select
Arch : noarch
Version : 3.3.1.0
Release : 10
Size : 29 k
Repo : installed
From repo : HDF-3.3-repo-152
Summary : hdf-select Distro select package
License : APL2
Description : hdf-select-3.3.1.0 select package

Uploading Enabled Repositories Report
Loaded plugins: langpacks, product-id, subscription-manager
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#

[#10]

@pritam_konar 
Yes hdf-select might be installed. However we you have mentioned earlier that when you tried to run it manually still you noticed this error:  which means it might not be installed properly.   

[root@nladfmrvu12 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found

[root@nladfmrvu11 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found

Thats why try to do a yum reinstall once.

# yum reinstall hdf-select

What i meant to say is .... hdf-select might not be able to determine the packages.  hdf-select usualy looks inside the "/usr/hdf/" directory and checks the versions and the "current" ..

So do you see any version for hdf components when you run the hdf-select command?

# hdf-select

If not then please try to set the versions and then see if it works?

# hdf-select set nifi 3.3.1.0-10
# hdf-select set nifi-registry 3.3.1.0-10
# hdf-select set nifi-toolkit 3.3.1.0-10

[#11]

After re-installing hdf-select, i am getting the below error while setting the nifi version:-

[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# hdf-select
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# hdf-select set nifi 3.3.1.0-10
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
[root@nladfmrvu12 ~]#

[#12]

@pritam_konar 

Your latest error is :

[root@nladfmrvu12 ~]# hdf-select
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?

This indicates that you are using "Python3" which needs the print statements to be inside parentheses. 

You should be using Python 2.7.x to avoid this error OR you will have to manually update the hdf-select  script to change all 'print' statements with 'print()'

Example:

Old: print "The answer is", 2*2
New: print("The answer is", 2*2)

You can see this behaviour change in Python3 in the following doc: https://docs.python.org/3/whatsnew/3.0.html

The print statement has been replaced with a print() function, with keyword arguments to replace most of the special syntax of the old print statement (PEP 3105).

.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Better to add new Community Topic/Threads for completely unrelated errors that way it helps more users.

[#13]

Could you share the nifi log under 

/var/log/nifi/