Member since
09-29-2015
362
Posts
242
Kudos Received
63
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1312 | 03-14-2019 01:00 PM | |
1586 | 01-23-2019 04:19 PM | |
7187 | 01-15-2019 01:59 PM | |
4530 | 01-15-2019 01:57 PM | |
10582 | 12-06-2018 02:01 PM |
10-17-2017
04:54 PM
@Neha G In a kerberized cluster there are 2 types of keytabs or principals headless and service principals. Headless principals are not bound to a specific host or node and are presented like @ SRV.COM Service principals are bound to a specific service and host or node, and are presented like with syntax: /@ SRV.COM So when you initialize the hdfs.headless.keytab is as DoAs so the user will take hdfs permissions
... View more
10-14-2017
08:19 PM
1 Kudo
@Mamta Chawla You cannot use the MIT Kerberos kadmin to create accounts in an Active Directory. That tool is only for use with the MIT KDC. To create accounts in an Active Directory, you will need to use Active Directory-specific tools. However, you can use a similar process that Ambari uses to create accounts in an Active Directory and then manually build the keytab files. This requires the use of the OpenLDAP ldapadd and ldapmodify tools as well as the ktutil command. See https://community.hortonworks.com/articles/82544/how-to-create-ad-principal-accounts-using-openldap.html. On top of this, you will need to make sure your krb5.conf file is correct in order for you to test it out. The krb5.conf file is not needed to create the AD account if you are using LDAP to do the work.
... View more
06-23-2017
09:40 AM
Yikes... it appears that I had an error in the JAAS config that I posted. It was a typo on my part. However, I am glad you found the issue and fixed it. I accidentally had useKeyTab=false where the proper value was supposed to be useKeyTab=true
My apologies.
... View more
11-15-2018
02:16 PM
I am not exactly sure all of the tasks that the Ambari agent reset operation performs. It seems like it will do a lot more than just clean up the existing certs - if it does that at all. However, it will not perform any of the Ambari server-side tasks. So at lest the Ambari server steps from the article need to take place manually.
... View more
06-12-2017
03:03 PM
There are certain circumstances where the Ambari CA needs to be recreated. Maybe it was corrupted, maybe Ambari's CA certificate expired, etc.... This how-to, assumes that the certificates signed by the Ambari CA are replaceable - which is generally the case for certificates used by Ambari agents for 2-way SSL connections. In the end, the Ambari server and all the agents will be restarted, causing a new CA certificate to be created along with new SSL certificates for each of the Ambari agents. On the Ambari server:
Stop the Ambari server Backup /var/lib/ambari-server/keys and it child directories Delete the following files from /var/lib/ambari-server/keys ca.key ca.csr ca.crt pass.txt keystore.p12 *.csr *.crt Delete the following files from /var/lib/ambari-server/keys/db index.txt.old index.txt.attr.old serial.old Truncate the following files from /var/lib/ambari-server/keys/db index.txt index.txt.attr Edit the following files from /var/lib/ambari-server/keys/db serial set the contents to be exactly 00
Delete all files under /var/lib/ambari-server/keys/db/newcerts Restart Ambari server. On each Ambari agent host: Stop the Ambari agent Backup /var/lib/ambari-agent/keys and it child directories Delete the following files from /var/lib/ambari-agent/keys ca.crt *.crt *.csr *.key Restart Ambari agent After restarting the Ambari server, the following (or similar) entries should be seen in the /var/log/ambari-server/ambari-server.log file: 12 Jun 2017 14:38:19,606 INFO [main] ShellCommandUtil:63 - Command openssl genrsa -des3 -passout pass:**** -out /var/lib/ambari-server/keys/ca.key 4096 was finished with exit code: 0 - the operation was completely successfully
.
12 Jun 2017 14:38:19,640 INFO [main] ShellCommandUtil:63 - Command openssl req -passin pass:**** -new -key /var/lib/ambari-server/keys/ca.key -out /var/lib/ambari-server/keys/ca.csr -batch was finished with exit code: 0 - the o
peration was completely successfully.
12 Jun 2017 14:38:19,683 INFO [main] ShellCommandUtil:63 - Command openssl ca -create_serial -out /var/lib/ambari-server/keys/ca.crt -days 365 -keyfile /var/lib/ambari-server/keys/ca.key -key **** -selfsign -extensions jdk7_ca -config /var/lib/ambari-server/keys/ca.config -batch -infiles /var/lib/ambari-server/keys/ca.csr was finished with exit code: 0 - the operation was completely successfully.
12 Jun 2017 14:38:19,701 INFO [main] ShellCommandUtil:63 - Command openssl pkcs12 -export -in /var/lib/ambari-server/keys/ca.crt -inkey /var/lib/ambari-server/keys/ca.key -certfile /var/lib/ambari-server/keys/ca.crt -out /var/lib/ambari-server/keys/keystore.p12 -password pass:**** -passin pass:****
was finished with exit code: 0 - the operation was completely successfully.
12 Jun 2017 14:38:19,708 INFO [main] ShellCommandUtil:63 - Command find /var/lib/ambari-server/keys -type f -exec chmod 700 {} + was finished with exit code: 0 - the operation was completely successfully.
12 Jun 2017 14:38:19,708 INFO [main] ShellCommandUtil:63 - Command chmod 600 /var/lib/ambari-server/keys/pass.txt was finished with exit code: 0 - the operation was completely successfully.
....
12 Jun 2017 14:52:53,797 INFO [qtp-ambari-agent-34] CertificateManager:200 - Signing agent certificate
12 Jun 2017 14:52:53,800 INFO [qtp-ambari-agent-34] CertificateManager:220 - Validating agent hostname: c6401.ambari.apache.org
12 Jun 2017 14:52:53,800 INFO [qtp-ambari-agent-34] CertificateManager:232 - Verifying passphrase
12 Jun 2017 14:52:53,849 INFO [qtp-ambari-agent-34] ShellCommandUtil:63 - Command openssl ca -config /var/lib/ambari-server/keys/ca.config -in /var/lib/ambari-server/keys/c6401.ambari.apache.org.csr -out /var/lib/ambari-server/keys/c6401.ambari.apache.org.crt -batch -passin pass:**** -keyfile /var/lib/ambari-server/keys/ca.key -cert /var/lib/ambari-server/keys/ca.crt was finished with exit code: 0 - the operation was completely successfully.
After restarting the Ambari agent, the following (or similar) entires should be seen in the /var/log/ambari-agent/ambari-agent.log file: INFO 2017-06-12 14:52:53,625 security.py:55 - Server require two-way SSL authentication. Use it instead of one-way...
INFO 2017-06-12 14:52:53,625 security.py:179 - Server certicate not exists, downloading
INFO 2017-06-12 14:52:53,625 security.py:202 - Downloading server cert from https://localhost:8440/cert/ca/
INFO 2017-06-12 14:52:53,693 security.py:187 - Agent key not exists, generating request
INFO 2017-06-12 14:52:53,693 security.py:258 - openssl req -new -newkey rsa:1024 -nodes -keyout "/var/lib/ambari-agent/keys/c6401.ambari.apache.org.key" -subj /OU=c6401.ambari.apache.org/ -out "/var/lib/ambari-agent/keys/c6401.ambari.apache.org.csr"
INFO 2017-06-12 14:52:53,736 security.py:195 - Agent certificate not exists, sending sign request
INFO 2017-06-12 14:52:53,855 security.py:93 - SSL Connect being called.. connecting to the server
INFO 2017-06-12 14:52:53,933 security.py:77 - SSL connection established. Two-way SSL authentication completed successfully.
... View more
Labels:
06-01-2017
01:41 PM
Thanks Graham and Robert. This is helpful.
... View more
06-12-2017
03:24 PM
@amarnath reddy pappu @mqureshi, @Kuldeep Kulkarni, @Gerd Koenig, @Andrew Ryansmaple-mapreduce-job-error.txtmapreduce-error-in-hive-wiht-beeline.txt
We have enabled SSL/TLS on HDP cluster by following @amarnath reddy pappu blog :
https://community.hortonworks.com/articles/52875/enable-https-for-hdfs.html and HDP documentation.
Almost all service opening on Https defined port. But Only issue we are currently facing is : MAP
REDUCE JOBS ARE NOT LAUNCHING
We use hive through beeline connector. While executing query we receive error :
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty (state=08S01,code=1) Tried a Sample Map reduce Job alone as well. That also got failed. Error is long so attaching here.
I would appreciate your help. 🙂
... View more
03-28-2017
03:17 AM
1 Kudo
@Elvis Zhang You can avoid Ambari UI to ask password for kadmin. You can store that kadmin credential to ambari store. https://community.hortonworks.com/articles/42927/adding-kdc-administrator-credentials-to-the-ambari.html curl -H "X-Requested-By:ambari" -u admin:admin -X PUT -d '{ "Credential" : { "principal" : "admin/admin@EXAMPLE.COM", "key" : "pwd$hwx", "type" : "persisted" } }' http://ambari.example.com:8080/api/v1/clusters/c1/credentials/kdc.admin.credential . Above will require a keystore setup and then "[2] Encrypt passwords stored in ambari.properties file." on ambari-server setup-security wizard. Using this way you can avoid entering the kadmin credentials everytime on ambari.
... View more
03-29-2017
12:59 PM
Just to sum it up: I have now chosen to place some regex in the auth-to-local rules to match exactly those hosts, which are used in a certain cluster. While this adds operations overhead, it will make the cluster more secure. The guys of Cloudera have a good summary about that in their documentation: https://www.cloudera.com/documentation/enterprise/5-9-x/topics/sg_auth_to_local_isolate.html
... View more