From further research, it looks like Suse has support for CGROUPS, as shown by this link: https://www.suse.com/documentation/opensuse114/book_tuning/data/sec_tuning_cgroups_usage.html I am still wondering is this supported by the HDP platform for Suse? I don't a statement one way or the other in the HWX documentation. ... View more

Hi @Deepesh - mouse over doesn't provide any information. This is in Ambari under Advanced hive-env, so it is not a custom value. I see this in both of the customer's clusters, plus in a cleanly installed lab environment matching their versions. I also don't see any evidence of this setting in any of the hive-env.sh files. Any thoughts? ... View more

Hi @naveen sangam -- yes, since you are using a self-signed certificate you generated with keytool -genkey, you will need to export the certificate into a .cer file, and then import it into your truststore file on the client. All of this can be accomplished with the keytool command. Be sure when you import the certificate into your truststore that it shows up as a "TrustedCertEntry". ... View more

Hi @hardik desai - I've learned to take a broad view of messages about obtaining passwords for Kerberos. Here's a suggested list of things to check: Verify file permissions and ownership of the /etc/security/keytabs/*.keytab files is correct (mentioned above, see reference below). Keep in mind the local Linux user that runs a service must be able to access the appropriate keytab file. Verify you can manually kinit as the user principal (also mentioned above). I recommend performing an "su" to the local Linux account first (like "su hdfs" and then kinit to check the hdfs principal). Follow the steps mentioned above if you need help with kinit command. Use the klist -ket /etc/security/keytabs/<keytab-name> to verify your principal names are set correctly in the configuration. Ambari automated installation will handle all this for you, but a manual installation should be double checked. Verify your version of Java is correct for your HDP distribution. Supported versions are listed in the installation docs from docs.hortonworks.com under Getting Ready / Meet Minimum System Requirements / JDK requirements. Also note depending on your Java distribution you may have to install the Java Cryptography Extensions (JCE) that matched your JRE version. OpenJDK already includes what it needs, but Oracle JDK requires an additional installation. Older versions of HDP (like HDP 2.2 and I think the earlier HDP 2.3 versions) had some issues with certain Java / JCE combinations that caused problems with Kerberos. Again a double check of supported Java versions is recommended to be sure yours isn't too old or too new. For reference, here's the permissions on the directory and some of the keytabs from a working installation: [root@m1 ~]# ll /etc/security | grep keytabs drwxr-xr-x. 2 root root 4096 Mar 6 11:34 keytabs [root@m1 ~]# ll /etc/security/keytabs -r--------. 1 hdfs hadoop 186 Mar 5 17:55 dn.service.keytab -r--r-----. 1 hdfs hadoop 156 Mar 5 17:55 hdfs.headless.keytab -r--r-----. 1 yarn hadoop 190 Mar 5 17:55 hive.llap.zk.sm.keytab -r--r-----. 1 hive hadoop 190 Mar 5 17:55 hive.service.keytab -r--------. 1 mapred hadoop 188 Mar 5 17:55 jhs.service.keytab -r--------. 1 hdfs hadoop 186 Mar 5 21:51 jn.service.keytab -r--------. 1 yarn hadoop 186 Mar 5 17:55 nm.service.keytab -r--------. 1 hdfs hadoop 186 Mar 5 17:55 nn.service.keytab -r--------. 1 yarn hadoop 186 Mar 5 17:55 rm.service.keytab -r--r-----. 1 ambari-qa hadoop 166 Mar 5 17:55 smokeuser.headless.keytab -r--r-----. 1 root hadoop 190 Mar 5 17:55 spnego.service.keytab -r--------. 1 yarn hadoop 190 Mar 5 17:55 yarn.service.keytab -r--------. 1 zookeeper hadoop 200 Mar 5 17:55 zk.service.keytab Good luck! ... View more

Hi @naveen sangam -- SSL config is tricky at first. Here are some pointers that will hopefully get you on the path: The keytool -genkey command above is creating a self-signed certificate. Another option is to follow your company's process to create a company signed certificate, usually thru your IT department, but that can take time. Self signed are good for non-prod environment, but I recommend company signed SSL cert for prod environments. Once you have a valid certificate, it must be installed on the server. That's the Hive configuration steps you listed above, but I think you have a mistake. Hive should path to a keystore file (not a truststore) like "hiveserver2.jks". You can use the "keytool -import" command to create the keystore file if needed. Think of it this way: a server secures communication using a certificate that's saved in a keystore. The client trusts that certificate using what is saved in the client's truststore. Keystore = SSL server, truststore = SSL client. In our case this will be Hiveserver 2 (server / keystore) and beeline (client / truststore). Please note: the files are all considered "jks" or "java keystore" files, but they have different uses and different entries contained within. So there are keystore files that are used as keystores, and there are keystore files used as truststores. Whoever named all this should be taken out back and shot. To check if your keystore is properly setup, run the command "keytool -list -v -keystore <keystore-filename.jks>" and enter password. Look for an entry for your certificate, and the entry type should read "PrivateKeyEntry" with valid expiration date. Now the Hive client (like beeline) must trust that server certificate. You will need to import the self-signed certificate, or your company's CA certificate (depending on your certificate type) into a truststore, like "cacerts". To check if your truststore is properly setup, run the command "keytool -list -v -keystore <truststore-filename.jks>" and enter password. Look for an entry for your certificate, and the entry type should read "TrustedCertEntry" with valid expiration date. Java has a default truststore file that it uses, called "cacerts", and this is located within your java installation directory. You can generally sniff it out by running "locate cacerts". If you import into this location, beeline should find the CA trusted cert entry with no problem. I believe the default location is /etc/pki/java/cacerts, and default password is "changeit". However, you can also create another truststore file and save it anywhere your beeline user can read the file. Since this isn't the truststore that's built into java, you have to tell beeline where this file lives by adding the path and password to the truststore to your connection string: jdbc:hive2://<host>:<port>/<database>;ssl=true;sslTrustStore=<path-to-truststore>;trustStorePassword=<password> Keytool is your best friend through all of this, so get familiar with it. Here is a link to some of the most common keytool commands: https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html Hope this helps -- Eddie ... View more

Good info -- thanks David ... View more

Hi Norman -- I remember working through a similar issue a while back. Here's what I remember, and hopefully it'll get you closer: After installation of the MIT Kerberos client, there's a krb5.ini file under c:\program data\MIT\Kerberos5. The "Program Data" directory is hidden, so you'll need to unhide it, or manually type it to find the files. You'll need to edit this INI file to add your REALM info for the MIT KDC. There were also some changes needed to the browser to pick up on MIT Kerberos client, plus to allow access to the URIs you are trying to reach. Here's a link to some of the details for Firefox: http://www.microhowto.info/howto/configure_firefox_to_authenticate_using_spnego_and_kerberos.html You can probably find similar info on other browsers by searching on SPNEGO + Kerberos + browser name Hope this helps -- Eddie ... View more

Great article Wes! ... View more

Another possible cause for this are the following two Ranger settings: ranger.service.https.attrib.clientAuth ranger.service.https.attrib.client.auth Change these settings from "false" to "want" if needed ... View more