SYMPTOM: User has latest HDP integrated with kerberos. While starting the datanode user gets the message: Login failure for dn/host1@EXAMPLE.NET from keytab /etc/security/keytabs/dn.service.keytab. But the principal is dn/host1.bc@EXAMPLE.NET Where host1 is the hostname of the datanode host and EXAMPLE.NET is the REALM name. ERROR: The output of klist command is as below - $klist -kt /etc/security/keytabs/dn.service.keytab Keytab name: FILE:/etc/security/keytabs/dn.service.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 0 12/21/2016 10:38:13 dn/host1.bc@EXAMPLE.NET In logs it shows - dn/host1@EXAMPLE.NET Where as it should show - dn/host1.bc@EXAMPLE.NET ROOT CAUSE: This is issue with entries in /etc/host file. RESOLUTION: User has below entry in /etc/hosts file - <ipaddress> <hostname> <FQDN> <FQDN> Now the order is changed to <ipaddress> <FQDN> <hostname> <FQDN> Which resolved the issue. ... View more

Question: We added the following settings to our knox topology so that knox may search in ldap for the groups that a user is in, so that we can efficiently control knox permissions with ranger. <!-- changes needed for group sync--> <param> <name>main.ldapGroupContextFactory</name> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value> </param> <param> <name>main.ldapRealm.authorizationEnabled</name> <value>true</value> </param> <param> <name>main.ldapRealm.groupSearchBase</name> <value>OU=Infrastructure,OU=EXAMPLE,DC=test,DC=local</value> </param> <param> <name>main.ldapRealm.groupObjectClass</name> <value>group</value> </param> <param> <name>main.ldapRealm.groupIdAttribute</name> <value>cn</value> </param> Our search base and attribute settings are the same as those we have in our hadoop.security.ldap.group.mapping config, and for ranger. But in knox we get an error in the gateway.log (attached) complaining about the size of the results. Normally the results are paginated, but it seems like knox is trying to get all of them at once. What do we need to change to make this work? ERROR: Knox was searching in OU which has 3000 groups. Below was setup - 1. Env has KNOX = AD integration in place 2. Env currently have 3000+ groups in AD and current limit for a search, without pagination, is 500. 3. User has Ranger -Knox plugin enabled. 4. User level policy for knox -ranger plugin works whereas it does not works if groups are specified in policy. ROOT CAUSE: This is a BUG - https://issues.apache.org/jira/browse/KNOX-644 and is Fixed in Knox 0.10 version. RESOLUTION: Adding below property resolved the issue - <param> <name>main.ldapRealm.authorizationEnabled</name> <value>false</value> </param> ... View more

Create the self signed certificate and add it to a keystore file using: keytool -genkey -alias example.com -keyalg RSA -keystore keystore.jks -keysize 2048 2. List the keystore entries to verify that the certificate was added. Note that a keystore can contain multiple such certificates: keytool -list -keystore keystore.jks 3. Export this certificate from keystore.jks to a certificate file: keytool -export -alias example.com -file example.com.crt -keystore keystore.jks 4. Add this certificate to the client's truststore to establish trust: keytool -import -trustcacerts -alias example.com -file example.com.crt -keystore truststore.jks 5. Verify that the certificate exists in truststore.jks: keytool -list -keystore truststore.jks 6. Set hive.server2.thrift.sasl.qop=auth in HS2 configs Then start HiveServer2, login with user->kinit->beeline and try to connect with beeline using: !connect jdbc:hive2://<hs2_hostname>:10001/default;principal=<hive_principal>;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=<truststore_file_path>;trustStorePassword=<truststore_password> ... View more

SYMPTOM: Created a user in Ranger. User is visible in ranger DB but not visible in Ranger UI ERROR: Logged into mysql DB and executed below command - SELECT * FROM ranger.x_user where user_name in ('userA'); ==> This shows the user exist in x_user table. SELECT * FROM ranger.x_portal_user where user_name in ('userA'); ==> The user is also present in x_portal_user ROOT CAUSE: Suspected corruption on Ranger DB sometimes. RESOLUTION: Executing below command resolved the issue >INSERT INTO x_portal_user_role VALUES(NULL,'2016-09-09 00:00:00','2016-09-09 00:00:00',1,1,(SELECT id FROM x_portal_user WHERE login_id='XXXX'),'ROLE_USER',1); Replace XXXX with the login_id of the user ('userA') You can replace 'ROLE_USER' with 'ROLE_SYS_ADMIN' if you want it to be an admin ... View more

SYMPTOM: HDP upgrade was failed on HDFS startup. Namenode was not able to start and below were log messages - ERROR: From the detailed logs we see below error - ROOT CAUSE: The above log clearly indicates"ClassNotFound" error. Customer has integrated customer jar in hadoop which was causing the issue. RESOLUTION: There was custom jar which was already in place with Previous HDP version [located in path - /usr/hdp/2.4.3.0-37/hadoop/sas*.jar ]. Adding the jar from earlier version to the upgraded version path [ie. /usr/hdp/2.5.3.0-37/hadoop/] resolved the issue. [Note: There was custom implementation of SAS with hadoop for the setup and hence the custom jars were present in path mentioned above ie. /usr/hdp/2.4.3.0-37/hadoop/sas*.jar. Default setup never includes any custom app/jar implementation with hadoop. It usually refers or org.apache.hadoop class.] ... View more