Created 12-04-2018 06:29 PM
Hi,
The hdfs encryption guide for HDP 3.0.1 states "In Ambari, replace the current value of dfs.permissions.superusergroup
with the group name “operator”" however, this option is not available from Ambari, so it looks like the fix applied in 2.6.0 is not working for 2.7.1.
I also checked the workaround suggested in this community post but commenting or even removing the tag does not change anything from Ambari's UI perspective.
Versions used:
ambari 2.7.1.0-169
HDP 3.0.1.0-187
Does anyone have other workaround or fix to this?
Thank you in advance.
Created 12-10-2018 03:21 PM
Hi @Diego J,
I guess thats a bug in Ambari.
can you please use this to update the value of dfs.permissions.superusergroup
.
/var/lib/ambari-server/resources/scripts/configs.py -l <AMBARI_HOSTNAME> -t 8080 -u <USER_NAME> -p <PASSWORD> -a <GET/SET/DELETE> -n <CLUSTER_NAME> -c <CONFIG_TYPE> -k <KEY> -v <VALUE>
for ex :
[root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py --help Usage: configs.py [options] Options: -h, --help show this help message and exit -t PORT, --port=PORT Optional port number for Ambari server. Default is '8080'. Provide empty string to not use port. -s PROTOCOL, --protocol=PROTOCOL Optional support of SSL. Default protocol is 'http' -a ACTION, --action=ACTION Script action: <get>, <set>, <delete> -l HOST, --host=HOST Server external host name -n CLUSTER, --cluster=CLUSTER Name given to cluster. Ex: 'c1' -c CONFIG_TYPE, --config-type=CONFIG_TYPE One of the various configuration types in Ambari. Ex: core-site, hdfs-site, mapred-queue-acls, etc. To specify credentials please use "-e" OR "-u" and "-p'": -u USER, --user=USER Optional user ID to use for authentication. Default is 'admin' -p PASSWORD, --password=PASSWORD Optional password to use for authentication. Default is 'admin' -e CREDENTIALS_FILE, --credentials-file=CREDENTIALS_FILE Optional file with user credentials separated by new line. To specify property(s) please use "-f" OR "-k" and "-v'": -f FILE, --file=FILE File where entire configurations are saved to, or read from. Supported extensions (.xml, .json>) [root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py -l asnaik1 -t 8080 -u admin -p admin -a set -n asnaik -c hdfs-site -k dfs.permissions.superusergroup -v hdfs,operator 2018-12-10 15:19:00,604 INFO ### Performing "set": 2018-12-10 15:19:00,604 INFO ### new property - "dfs.permissions.superusergroup":"hdfs,operator" 2018-12-10 15:19:00,663 INFO ### on (Site:hdfs-site, Tag:version1543379050314) 2018-12-10 15:19:00,675 INFO ### PUTting json into: doSet_version1544455140675467.json 2018-12-10 15:19:00,767 INFO ### NEW Site:hdfs-site, Tag:version1544455140675467 [root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py -l asnaik1 -t 8080 -u admin -p admin -a get -n asnaik -c hdfs-site -k dfs.permissions.superusergroup |grep -i dfs.permissions.superusergroup "dfs.permissions.superusergroup": "hdfs,operator",
Adding to this,
If you want to enable it in ambari UI,we can add it by following the steps
For Ambari-2.6.x versions :
cd /var/lib/ambari-server/resources/common-services/HDFS/2.1.0.2.0/configuration/ vi hdfs-site.xml remove the property as mentioned in : https://github.com/apache/ambari/blob/8e0f782efd4694028b598106e68ebe2a1c7c0a2e/ambari-server/src/mai... ambari-server restart
For Ambari-2.7.x versions :
cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services/HDFS/configuration/ vi hdfs-site.xml remove the property as mentioned in : https://github.com/apache/ambari/blob/8e0f782efd4694028b598106e68ebe2a1c7c0a2e/ambari-server/src/mai... ambari-server restart
Created 12-10-2018 03:21 PM
Hi @Diego J,
I guess thats a bug in Ambari.
can you please use this to update the value of dfs.permissions.superusergroup
.
/var/lib/ambari-server/resources/scripts/configs.py -l <AMBARI_HOSTNAME> -t 8080 -u <USER_NAME> -p <PASSWORD> -a <GET/SET/DELETE> -n <CLUSTER_NAME> -c <CONFIG_TYPE> -k <KEY> -v <VALUE>
for ex :
[root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py --help Usage: configs.py [options] Options: -h, --help show this help message and exit -t PORT, --port=PORT Optional port number for Ambari server. Default is '8080'. Provide empty string to not use port. -s PROTOCOL, --protocol=PROTOCOL Optional support of SSL. Default protocol is 'http' -a ACTION, --action=ACTION Script action: <get>, <set>, <delete> -l HOST, --host=HOST Server external host name -n CLUSTER, --cluster=CLUSTER Name given to cluster. Ex: 'c1' -c CONFIG_TYPE, --config-type=CONFIG_TYPE One of the various configuration types in Ambari. Ex: core-site, hdfs-site, mapred-queue-acls, etc. To specify credentials please use "-e" OR "-u" and "-p'": -u USER, --user=USER Optional user ID to use for authentication. Default is 'admin' -p PASSWORD, --password=PASSWORD Optional password to use for authentication. Default is 'admin' -e CREDENTIALS_FILE, --credentials-file=CREDENTIALS_FILE Optional file with user credentials separated by new line. To specify property(s) please use "-f" OR "-k" and "-v'": -f FILE, --file=FILE File where entire configurations are saved to, or read from. Supported extensions (.xml, .json>) [root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py -l asnaik1 -t 8080 -u admin -p admin -a set -n asnaik -c hdfs-site -k dfs.permissions.superusergroup -v hdfs,operator 2018-12-10 15:19:00,604 INFO ### Performing "set": 2018-12-10 15:19:00,604 INFO ### new property - "dfs.permissions.superusergroup":"hdfs,operator" 2018-12-10 15:19:00,663 INFO ### on (Site:hdfs-site, Tag:version1543379050314) 2018-12-10 15:19:00,675 INFO ### PUTting json into: doSet_version1544455140675467.json 2018-12-10 15:19:00,767 INFO ### NEW Site:hdfs-site, Tag:version1544455140675467 [root@asnaik-asnaik1 ~]# /var/lib/ambari-server/resources/scripts/configs.py -l asnaik1 -t 8080 -u admin -p admin -a get -n asnaik -c hdfs-site -k dfs.permissions.superusergroup |grep -i dfs.permissions.superusergroup "dfs.permissions.superusergroup": "hdfs,operator",
Adding to this,
If you want to enable it in ambari UI,we can add it by following the steps
For Ambari-2.6.x versions :
cd /var/lib/ambari-server/resources/common-services/HDFS/2.1.0.2.0/configuration/ vi hdfs-site.xml remove the property as mentioned in : https://github.com/apache/ambari/blob/8e0f782efd4694028b598106e68ebe2a1c7c0a2e/ambari-server/src/mai... ambari-server restart
For Ambari-2.7.x versions :
cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services/HDFS/configuration/ vi hdfs-site.xml remove the property as mentioned in : https://github.com/apache/ambari/blob/8e0f782efd4694028b598106e68ebe2a1c7c0a2e/ambari-server/src/mai... ambari-server restart
Created 12-11-2018 04:14 AM
Hi @Diego J,
Just to add to this. this feature was caused due to fixing of https://issues.apache.org/jira/browse/AMBARI-22086 where the developer makes the dfs.permissions.superusergroup a group property so that ambari doesn't show it in any tab.
If you want to show this in ambari. we can go to : /var/lib/ambari-server/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml
( i am not sure if this is the path for HDP-3.x versions, please use find command in /var/lib/ambari-server/resources/common-services/ to figure out your path )
and remove : <property-type>GROUP</property-type> for : dfs.permissions.superusergroup
refer to fix of this issue here : https://reviews.apache.org/r/62669/diff/1#1
Please accept answer if you find this helpful.
Created 12-11-2018 05:13 PM
Hi @Akhil S Naik, thank you for your answers. Using the config.py script works good, I'm only going to add that you need to use "-s https" when security has been already configured, like this:
/var/lib/ambari-server/resources/scripts/configs.py -l <ambari-server> -t 8443 -s https -u <admin-user> -p <passwd> -a set -n <cluster-name> -c hdfs-site -k dfs.permissions.superusergroup -v hdfs,<superuser_group>
And for the second answer, I'm going to add that in HDP 3.0.1, the right path to remove the property <property-type>GROUP</property-type> is:
/var/lib/ambari-server/resources/stacks/HDP/3.0/services/HDFS/configuration/hdfs-site.xml