Created on
08-23-2019
01:18 AM
- last edited on
08-27-2019
07:21 AM
by
cjervis
When we are trying to implement Ranger Plugin in Nifi in our ambari clsuter, we are getting the attached error. We have configured Nifi in distributed mode in two different servers.Please find the attached error log.
Ambari Version :- 2.7.3.0
HDP Version :- 3.1.0.0
HDF version :- 3.3.1.0-10
Nifi Version:- 1.8.0.3.3.1.0-10
stderr:
2019-08-22 17:08:51,422 - Stack selector path does not exist
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 304, in <module>
Master().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 144, in start
setup_ranger_nifi(upgrade_type=None)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/setup_ranger_nifi.py", line 85, in setup_ranger_nifi
component_user_keytab=params.ranger_nifi_keytab if params.security_enabled else None, cred_lib_path_override = cred_lib_prefix_path, cred_setup_prefix_override = cred_setup_prefix_path)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/setup_ranger_plugin_xml.py", line 74, in setup_ranger_plugin
stack_version = get_stack_version(component_select_name)
File "/usr/lib/ambari-agent/lib/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/get_stack_version.py", line 84, in get_stack_version
'Unable to determine the current version because of a non-zero return code of {0}'.format(str(return_code)))
resource_management.core.exceptions.Fail: Unable to determine the current version because of a non-zero return code of 1
Created 08-23-2019 06:59 AM
Can anyone help here, please?
Created 08-26-2019 05:33 AM
Created 08-26-2019 02:57 PM
Looks like it is failing even before the nifi is getting started.
So can you please check and share what is the output of the following commands on the failing host?
# hdf-select | grep nifi
# cat /etc/redhat-release
Or please let us know what is the OS type or version? As we see that the error is coming at the execution of 'get_stack_version' so it will be good to know the "hdf-select" output and also output of the following directories/listing:
# ls -lart /usr/hdf
# ls -lart /usr/hdf/current/
Created 08-27-2019 12:41 AM
Nifi is installed in two servers. Please find the output from both of them:-
1) First Server:-
[root@nladfmrvu11 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# ls -lart /usr/hdf
total 4
drwxr-xr-x. 6 root root 66 Jul 29 15:21 3.3.1.0-10
drwxr-xr-x. 18 root root 4096 Aug 13 20:46 ..
drwxr-xr-x. 5 root root 48 Aug 21 11:17 .
drwxr-xr-x. 2 root root 17 Aug 21 11:18 None
drwxr-xr-x. 2 root root 36 Aug 23 15:55 current
[root@nladfmrvu11 ~]#
[root@nladfmrvu11 ~]# ls -lart /usr/hdf/current/
total 0
lrwxrwxrwx. 1 root root 24 Jul 30 11:50 nifi -> /usr/hdf/3.3.1.0-10/nifi
lrwxrwxrwx. 1 root root 32 Jul 30 11:52 nifi-toolkit -> /usr/hdf/3.3.1.0-10/nifi-toolkit
drwxr-xr-x. 5 root root 48 Aug 21 11:17 ..
drwxr-xr-x. 2 root root 36 Aug 23 15:55 .
[root@nladfmrvu11 ~]#
2) Second Server:-
[root@nladfmrvu12 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# ls -lart /usr/hdf
total 4
drwxr-xr-x. 17 root root 4096 Aug 1 10:04 ..
drwxr-xr-x. 5 root root 46 Aug 1 10:04 3.3.1.0-10
drwxr-xr-x. 2 root root 36 Aug 1 10:30 current
drwxr-xr-x. 5 root root 48 Aug 23 11:41 .
drwxr-xr-x. 2 root root 17 Aug 23 11:41 None
[root@nladfmrvu12 ~]# ls -lart /usr/hdf/current/
total 0
lrwxrwxrwx. 1 root root 24 Aug 1 10:11 nifi -> /usr/hdf/3.3.1.0-10/nifi
lrwxrwxrwx. 1 root root 32 Aug 1 10:12 nifi-toolkit -> /usr/hdf/3.3.1.0-10/nifi-toolkit
drwxr-xr-x. 2 root root 36 Aug 1 10:30 .
drwxr-xr-x. 5 root root 48 Aug 23 11:41 ..
[root@nladfmrvu12 ~]#
Created on 08-27-2019 01:00 AM - edited 08-27-2019 01:01 AM
How did you install NiFi on those two hosts? Via Ambari or Manually? Looks like somehow the "hdf-select" installation is missing on the Nifi Host.
I was expecting to see the output from the following command:
# hdf-select | grep nifi
nifi - 3.3.1.0-10
nifi-registry - 3.3.1.0-10
nifi-toolkit - 3.3.1.0-10
.
Looks like hdf-select is not installed on your host.
Can you please try this to see if it works?
Install the hdf-select manually as following:
# yum info hdf-select
# yum clean all
# yum install hdf-select
Then set the correct version for the stack.
# hdf-select set nifi 3.3.1.0-10
# hdf-select set nifi-registry 3.3.1.0-10
# hdf-select set nifi-toolkit 3.3.1.0-10
.
Then try to enable ranger . nifi plugin again.
.
If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Created 08-28-2019 12:45 AM
Thanks @jsensharma for the reply. I just want a confirmation:-
If i run the command :-
yum clean all
will it remove all the installed yum packages, or it will remove only the 'hdf-select' packages?
Created on 08-28-2019 12:47 AM - edited 08-28-2019 12:51 AM
"#yum clean all" will not delete any package from the host. It will just clean the yum cache header/packages.
# yum clean packages Eliminate any cached packages from the system. Note that packages are not automatically deleted after they are downloaded.
# yum clean headers Eliminate all of the header files which yum uses for dependency resolution.
# yum clean all Runs yum clean packages and yum clean headers as above.
Created on 08-28-2019 02:21 AM - edited 08-28-2019 02:27 AM
I thing hdf-select is installed already in both the cluster. Please find the output below and confirm:-
Please note , we installed Nifi via apache ambari.
[root@nladfmrvu11 ~]# yum info hdf-select
Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
Installed Packages
Name : hdf-select
Arch : noarch
Version : 3.3.1.0
Release : 10
Size : 29 k
Repo : installed
From repo : HDF-3.3-repo-152
Summary : hdf-select Distro select package
License : APL2
Description : hdf-select-3.3.1.0 select package
Uploading Enabled Repositories Report
Loaded plugins: langpacks, product-id, subscription-manager
[root@nladfmrvu11 ~]#
[root@nladfmrvu12 ~]# yum info hdf-select
Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager
Installed Packages
Name : hdf-select
Arch : noarch
Version : 3.3.1.0
Release : 10
Size : 29 k
Repo : installed
From repo : HDF-3.3-repo-152
Summary : hdf-select Distro select package
License : APL2
Description : hdf-select-3.3.1.0 select package
Uploading Enabled Repositories Report
Loaded plugins: langpacks, product-id, subscription-manager
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#
Created 08-28-2019 02:48 AM
@pritam_konar
Yes hdf-select might be installed. However we you have mentioned earlier that when you tried to run it manually still you noticed this error: which means it might not be installed properly.
[root@nladfmrvu12 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
[root@nladfmrvu11 ~]# hdf-select | grep nifi
-bash: hdf-select: command not found
Thats why try to do a yum reinstall once.
# yum reinstall hdf-select
What i meant to say is .... hdf-select might not be able to determine the packages. hdf-select usualy looks inside the "/usr/hdf/" directory and checks the versions and the "current" ..
So do you see any version for hdf components when you run the hdf-select command?
# hdf-select
If not then please try to set the versions and then see if it works?
# hdf-select set nifi 3.3.1.0-10
# hdf-select set nifi-registry 3.3.1.0-10
# hdf-select set nifi-toolkit 3.3.1.0-10
Created 08-30-2019 05:47 AM
After re-installing hdf-select, i am getting the below error while setting the nifi version:-
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# hdf-select
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]#
[root@nladfmrvu12 ~]# hdf-select set nifi 3.3.1.0-10
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
[root@nladfmrvu12 ~]#
Created on 08-31-2019 02:18 PM - edited 08-31-2019 02:19 PM
Your latest error is :
[root@nladfmrvu12 ~]# hdf-select
File "/bin/hdf-select", line 216
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
This indicates that you are using "Python3" which needs the print statements to be inside parentheses.
You should be using Python 2.7.x to avoid this error OR you will have to manually update the hdf-select script to change all 'print' statements with 'print()'
Example:
Old: print "The answer is", 2*2
New: print("The answer is", 2*2)
You can see this behaviour change in Python3 in the following doc: https://docs.python.org/3/whatsnew/3.0.html
The print statement has been replaced with a print() function, with keyword arguments to replace most of the special syntax of the old print statement (PEP 3105).
.
.
If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Better to add new Community Topic/Threads for completely unrelated errors that way it helps more users.
Created 08-26-2019 09:16 AM
Could you share the nifi log under
/var/log/nifi/