Member since
11-05-2016
15
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2519 | 05-02-2017 06:08 PM |
05-02-2017
06:08 PM
Hi @Jacqualin jasmin - looking at the /etc/passwd in my lab, I see a mixture of service logins with ids of around 500 and others are over 1000. Hive specifically is less than 1000. I also looked at a larger secured production cluster, and all the service logins were over 1000. Looks like you have several options: (1) set min.user.id=500, but not sure this is advisable from security perspective, (2) create new accounts over 1000 and use those to launch your jobs, (3) white list the user somehow (not entirely sure how to do that), or (4) update the service accounts with higher numbers.
... View more
05-02-2017
01:01 PM
Hi @Jacqualin jasmin - Ambari generally takes care of this during installation. I just looked at a lab environment that was Ambari installed, and see the group numbers are all < 1000. I've worked with many customers, and never seen a case where this needed to be changed. As a result, I don't think you need to change this.
... View more
03-09-2017
07:48 PM
Hi @naveen sangam -- yes, since you are using a self-signed certificate you generated with keytool -genkey, you will need to export the certificate into a .cer file, and then import it into your truststore file on the client. All of this can be accomplished with the keytool command. Be sure when you import the certificate into your truststore that it shows up as a "TrustedCertEntry".
... View more
03-04-2017
03:13 PM
1 Kudo
Hi @naveen sangam -- SSL config is tricky at first. Here are some pointers that will hopefully get you on the path:
The keytool -genkey command above is creating a self-signed certificate. Another option is to follow your company's process to create a company signed certificate, usually thru your IT department, but that can take time. Self signed are good for non-prod environment, but I recommend company signed SSL cert for prod environments.
Once you have a valid certificate, it must be installed on the server. That's the Hive configuration steps you listed above, but I think you have a mistake. Hive should path to a keystore file (not a truststore) like "hiveserver2.jks". You can use the "keytool -import" command to create the keystore file if needed.
Think of it this way: a server secures communication using a certificate that's saved in a keystore. The client trusts that certificate using what is saved in the client's truststore. Keystore = SSL server, truststore = SSL client. In our case this will be Hiveserver 2 (server / keystore) and beeline (client / truststore).
Please note: the files are all considered "jks" or "java keystore" files, but they have different uses and different entries contained within. So there are keystore files that are used as keystores, and there are keystore files used as truststores. Whoever named all this should be taken out back and shot.
To check if your keystore is properly setup, run the command "keytool -list -v -keystore <keystore-filename.jks>" and enter password. Look for an entry for your certificate, and the entry type should read "PrivateKeyEntry" with valid expiration date.
Now the Hive client (like beeline) must trust that server certificate. You will need to import the self-signed certificate, or your company's CA certificate (depending on your certificate type) into a truststore, like "cacerts".
To check if your truststore is properly setup, run the command "keytool -list -v -keystore <truststore-filename.jks>" and enter password. Look for an entry for your certificate, and the entry type should read "TrustedCertEntry" with valid expiration date.
Java has a default truststore file that it uses, called "cacerts", and this is located within your java installation directory. You can generally sniff it out by running "locate cacerts". If you import into this location, beeline should find the CA trusted cert entry with no problem. I believe the default location is /etc/pki/java/cacerts, and default password is "changeit".
However, you can also create another truststore file and save it anywhere your beeline user can read the file. Since this isn't the truststore that's built into java, you have to tell beeline where this file lives by adding the path and password to the truststore to your connection string:
jdbc:hive2://<host>:<port>/<database>;ssl=true;sslTrustStore=<path-to-truststore>;trustStorePassword=<password>
Keytool is your best friend through all of this, so get familiar with it. Here is a link to some of the most common keytool commands: https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html Hope this helps -- Eddie
... View more
03-03-2017
10:11 AM
Good info -- thanks David
... View more
02-24-2017
08:40 PM
Great article Wes!
... View more
11-20-2016
03:48 AM
Hi @Karan Alang - this could be an issue with the DNS server setup. You might want to check your /etc/resolv.conf file and point it to an external DNS server, such as 8.8.8.8, and try again.
... View more