Member since
12-03-2016
91
Posts
27
Kudos Received
4
Solutions
07-08-2022
08:44 AM
1 Kudo
I thought it was something like this, but it was hard to believe! After 2 full installation of CDP base, it seems clear that CDP may have been a big step for the final user, but still has a lot of room for improvement in the sysadmin and devops side of the platform, specially in the way-back or recovery of many central configuration changes (kerberos, TLS) where it really sucks, even when compared with the now ancient HDP3.
... View more
06-20-2022
05:27 PM
1 Kudo
The solution above has been the accepted answer for a long time (since CM 6.x), but the procedure is not complete, at least for Cloudera Manager 7.6 with CDP Base 7.1.6 or above. After using Auto-TLS and needing to disable it because I needed to rename some nodes, I changed the web_tls and agent_tls settings in the Database, and changed the config.ini to disable TLS in all the agents' config and restarted all the cloudera-scm-* daemons (server/agents) and I was able to log to the UI and reach the hosts, but in "Management >> Security" the Cloudera Manager UI keeps showing "Auto-TLS is Enabled" in the menu bar, and if I try to add a new host it will fail with an error like "Unable to deploy host certificates" ... so it seems CM still has Auto-TLS enable somewhere/somehow. I have made a dump and double checked there isn't any TLS config parameter enabled in the DB, but there should be some other place from where CM assumes this is enabled.
... View more
08-19-2020
07:42 AM
If you use the $ZK_HOST defined in infra-solr-env.sh you should not need to include the /infra-solr prefix when getting the solrconfig.xml: source /etc/ambari-infra-solr/conf/infra-solr-env.sh /usr/lib/ambari-infra-solr/server/scripts/cloud-scripts/zkcli.sh -z $ZK_HOST \ -cmd getfile /configs/ranger_audits/solrconfig.xml solrconfig.xml The same applies when uploading the edited config.
... View more
04-03-2019
07:56 PM
According tto the documentation in this link: https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/data-operating-system/content/partition_a_cluster_using_node_labels.html the information in this articulo is wrong because if you assing all the nodes to 3 different exclusive Node Label partitions, AND you do not set the default Node Label in the default Queue then you will not have resources available to run any Job sent to the default queue without an explicit label.
... View more
12-24-2018
05:38 PM
I followed this with HDP 2.6.5 and the HBaseUI became accessible in the given URL but has many errors and links not working inside. I posted a question on how to fix this and then the answer resolving most of these issues here: https://community.hortonworks.com/questions/231948/how-to-fix-knox-hbase-ui.html You are welcome to test this and include these fixes in your article if you find it appropriate. Best regards
... View more
12-02-2018
04:13 PM
I found problems trying to validate the information in this article, and after doing my own research I have to say that it's inaccurate and in some aspects simply wrong. First of all it must be clear that the "ranger.ldap.*" set of parameters which should be configured under Ambari under "Ranger >> Config >> Advanced >> Ldap Settings" and "Advanced ranger-admin-site" are related only to Ranger Admin UI authentication and this has nothing to do with Ranger Usersync (different properties, different code, different daemon) which must be configured completely in the "Ranger >> Config >> Ranger User Info" section. This article mix the two set of LDAP related configurations for two different components in Ranger and this is confusing and not correct. All that I state here may be verified by looking at the source code of the following classes from the "ranger" and "spring-ldap" projects in GitHub: apache.ranger.security.handler.RangerAuthenticationProvider
org.springframework.security.ldap.authentication.BindAuthenticator
org.springframework.security.ldap.authentication.LdapAuthenticationProvider/AbstractLdapAuthenticationProvider Talking about "Ranger Admin LDAP authentication" the only two parameters you will need are the following: ranger.ldap.url = http://ldap-host:389
ranger.ldap.user.dnpattern = uid={0},ou=users,dc=example,dc=com<br> This is because the RangerAuthenticationProvider class first uses the method "getLdapAuthentication()" which in place will use the Spring's BindAuthenticator class with default parameters except from the previous properties. This will try to do a BIND as the DN obtained from "ldapDNPattern" replacing "{0}" with the username, and if this succeeds, the authentication will be granted to the user and nothing else is used!! The only case were the remaining "ranger.ldap.*" parameters are used is when "getLdapAuthentication()" fails, for example by setting the wrong value for "ldap.user.dnpattern" as in the example above, where the LDAP manager's DN will be used to do the Bind with the username's provided password. When the call to "getLdapAuthentication()" fails, Ranger will next try a call to the more specialized method "getLdapBindAuthentication()" and is this method that will use all the other "ranger.ldap.{bind|user|group}.*" properties! This time BindAuthenticator will be configured to bind with the provided "ranger.ldap.bind.dn/password" and will search for the user entry and their groups with the other properties, etc ... But even in this case there is another IMPORTANT error in the article above:
the pattern in "ranger.ldap.group.searchfilter" is wrong, because this is handled by the class DefaultLdapAuthoritiesPopulator and this will replace the '{0}' with the DistinguishedName (DN) of the user (NOT the username) and instead ist the '{1}' that will be replaced with the username. So, if you want to use the configuration above, you should replace {0} with {1} or even better just use "member={0}' as your group searchfilter. Regarding the "authorization" phase, in both the previous methods, if the authorization succeeds, then the group/role authorities for the user will be searched from LDAP (using Spring's DefaultLdapAuthoritiesPopulator class), but ONLY if both
rangerLdapGroupSearchBase AND rangerLdapGroupSearchFilter are defined
and not empty. But even in this case (I have still not tested this, but looking at the code It seems clear) I'm almost sure that the list of "grantedAuths"
obtained from LDAP are never used by Ranger because at the
end of both "getLDAP*Authentication()" methods the grantedAuths list is overwritten using the chain of calls to the following methods: authentication = getAuthenticationWithGrantedAuthority(authentication) >> List<GrantedAuthority> grantedAuths = getAuthorities(authentition.getName()); // pass username only >>>> roleList = (org.apache.ranger.biz.UserMgr)userMgr.getRolesByLoginId(username); //overwrite from Ranger DB I don't know if this is the desired behavior or it's a bug in the current RangerAuthenticationProvider that will be changed in the future (otherwise is not clear why to use the LdapAuthoritiesPopulator upstream), but it's the way it seems to be done right now. In conclusion, for Ranger Admin authentication, IF you just provides the right value for the "ranger.ldap.url" and "ranger.ldap.user.dnpattern" property, none
of the remaining "ranger.ldap.group.*" parameters will be used and the user roles
will be managed by Ranger from the "Admin UI -> Users" interface.
... View more
10-27-2018
09:58 PM
Nice work Greg, a wonderful, clear and very detailed article! Based on this
and other articles of yours in this forum, it seems clear that the people of
Hortonworks should seriously consider giving you a greater participation
in the elaboration and supervision of the (currently outdated, fuzzy, inexact and overall low qualilty) material presented in their
expensive official courses. The difference in quality between his work and the work published there is so great that it is sometimes rude. All my respects Mr!!
... View more
09-16-2018
12:41 AM
This article is really very useful but has a silly but confusing (specially for HDP newbies) error where all occurrences of "Ranger user id" and "Ranger Admin Server" must be replaced by "Atlas User ID" and "Atlas Admin Server" respectively.
... View more
09-13-2018
12:53 PM
At least for version 2.6.3 and above the section "Running import script on kerberized cluster" is wrong. You don't need to provide any of the options (properties) indicated (except maybe the debug one If you want to have debug output) because they are automatically detected and included in the script. Also at least in 2.6.5 a direct execution of the script in a Kerberized cluster will fail because of the CLASSPATH generated into the script. I had to edit this replacing many single JAR files by a glob inside their parent folder in order for the command to run without error. If you have this problem see answer o "Atlas can't see Hive tables" question.
... View more
08-31-2018
02:41 AM
This article is very illustrative but as most of the references in this topic doesn't addresses the problem of Linux users authentication against AD with the previous hosts and kerberos client configurations. For SSSD to work with AD you will have to setup your host's default krb5.keytabs to point to the AD principals (obtained when joining the host to the AD domain) and this is conflicting with the configuration described in the article where the host is supposed to be associated to the MIT KDC realm and default domain. How do we resolve this conflict?
... View more