Member since
05-09-2018
44
Posts
3
Kudos Received
0
Solutions
12-01-2018
11:45 AM
1 Kudo
Hi @aquilodran, I did try the steps which you recommend. But unfortunately it did not work. To make it work, I edited few properties in HDFS service allow anonymous = true http auth = simple (previous val was 'kerberos') Thank you!
... View more
11-30-2018
02:17 PM
1 Kudo
Hi,
We are running a self-signed certificate Ambari cluster (with HTTPS) and we also enabled the cluster with FreeIPA+Kerberos. Ambari URL: https://xxxx.xxxx.nm1:8443 (its not .com)
HDP: 3.0.1 (Latest)
After successfully integrating FreeIPA+Kerberos with Ambari cluster, we are unable to access few important GUIs such as Namenode UI, Resource Manager UI and Oozie UI. The error we are getting below is this:
HTTP ERROR 401
Problem accessing /index.html.
Reason:Authentication required
I've tried all possible scenarios to debug this error like running the following command in my MAC terminal but its of no use.
defaults write com.google.Chrome AuthServerWhitelist "*.REALM_NAME.COM"
defaults write com.google.Chrome AuthNegotiateDelegateWhitelist "*.REALM_NAME.COM"
I ran the same above command in Google Chrome console (option+command+j in MAC) and got this error: Uncaught SyntaxError: Unexpected identifier The following Keytabs are present in /etc/security/keytabs : kerberos.service_check.113018.keytab ambari.server.keytab spnego.service.keytab yarn-ats.hbase-regionserver.service.keytab yarn-ats.hbase-master.service.keytab smokeuser.headless.keytab oozie.service.keytab nn.service.keytab hive.service.keytab ams-monitor.keytab nm.service.keytab hive.llap.task.keytab hbase.headless.keytab spark.service.keytab spark.headless.keytab rm.service.keytab hdfs.headless.keytab ambari-infra-solr.service.keytab zk.service.keytab yarn.service.keytab yarn-ats.hbase-client.headless.keytab dn.service.keytab There is a valid ticket HDFS user as well but still unable to access the UI: hdfs@xxxxxxx:/etc/security/keytabs$ klist
Ticket cache: FILE:/tmp/krb5cc_1213
Default principal: nn/xxxxxx.xxxxxx.nm1@REALM.COM
Valid starting Expires Service principal
11/30/18 16:13:31 12/01/18 16:13:31 krbtgt/REALM.COM@REALM.COM
renew until 12/07/18 16:13:31 I also tried using "spnego.service.keytab" but still no use: root@xxxxxxx102:/etc/security/keytabs# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/xxxxxx102.xxxxx.nm1@REALM.COM
Valid starting Expires Service principal
11/30/18 17:23:38 12/01/18 17:23:38 krbtgt/REALM.COM@REALM.COM
renew until 12/07/18 17:23:38 Kindly provide your technical suggestions. It would be very helpful and highly appreciated. Should I disable the Kerberos HTTP authentication ? If yes, please guide me the same for NN, RM and Oozie URLs Thanks, Shesh Kumar
... View more
Labels:
11-21-2018
05:30 PM
Thank you so much Robert! Highly appreciate your views. I've one more doubt which I came across. It is about auto-renew of Kerberos ticket. As you know we have successfully integrated FreeIPA with Ambari cluster which also has IPA replication as well. I noticed that user's kerberos ticket is not auto-renewing even though they have a valid ticket. shesh.kumar@stg-ambarixenial001:~$ klist Ticket cache: FILE:/tmp/krb5cc_1193 Default principal: shesh.kumar@EXAMPLE.COM Valid starting Expires Service principal 11/18/18 18:15:37 11/19/18 18:15:34 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 11/25/18 18:15:34 As you can see above, the ticket is not auto-renewing. How can I make sure that kerberos ticket is auto-renewed once the user executes the "kinit" command. Let me show you what I have done from my side. I've added these 3 lines in /etc/sssd/sssd.conf file which is present in FreeIPA server (which don't have Hadoop client). krb5_lifetime = 120s krb5_renewable_lifetime = 150m krb5_renew_interval = 10s Will this work? Thanks, Shesh Kumar
... View more
11-18-2018
07:55 PM
Thanks for your suggestion. I really appreciate it. I have one more doubt. So If I have to remove/delete multiple users in IPA...say like 50 users, I will also need to login the server as root, switch to their user and fire "kdestroy" to remove the ticket cache? Won't this be too much of manual effort? What is the best practice that you recommend?
... View more
11-18-2018
02:18 PM
We are trying out FreeIPA and integrated the same to our Ambari Hadoop cluster (HDP v3.0.1). We are able to add users and provide them access to Hadoop with help of Kinit command. However, when deleting the users in FreeIPA GUI, the principal gets deleted. The deleted user's principal will not be there in the "kadmin" prompt when I do listprincs. But the user will still be having a valid ticket when he does "klist" and can access Hadoop even though the principal is removed. We cannot do "kdestory" manually. Typically, when users are removed in FreeIPA, the same users should not be able to access Hadoop as well. Can't FreeIPA handle kdestroy? Please provide your suggestions. Thanks, Shesh
... View more
Labels:
- Labels:
-
Apache Hadoop
10-30-2018
07:47 AM
Thank you! Will surely check the recommendation next time.
... View more
10-26-2018
03:04 AM
Thank you so much for your suggestion. However, I just happen to resolve this issue. Below I've shared my resolution. Please check and let me know what you think about it 🙂 If I face this situation again. Will try your suggestion the next time.
... View more
10-25-2018
09:04 PM
I somehow made it worked. Here's what I did: I added the following lines in the beginning and end of ALL the .sql files present in the path: "/usr/hdp/3.0.1.0-187/ranger-admin/db/mysql/patches/" and "/usr/hdp/3.0.1.0-187/ranger-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" SET FOREIGN_KEY_CHECKS=0; (Beginning of the .sql file) SET FOREIGN_KEY_CHECKS=1; (End of the .sql file) It takes lot of time to edit, add the lines and save and make sure you do this on host where Ranger was planned to installed Post this, delete the 'ranger' DB in MySQL, delete Ranger Service and again install Ranger in Ambari on same host where the .sql files are present that was edited. Thanks, Shesh Kumar
... View more
10-25-2018
02:25 AM
Hi, I'm unable to install Ranger Admin for the latest HDP version 3.0.1.0 (Latest) MySQL Version --> mysql Ver 15.1 Distrib 10.2.16-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 Error encountered: Error executing: INSERT INTO x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status) VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) SQLException : SQL state: 23000 com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452 2018-10-25 02:11:24,781[E] ranger_core_db_mysql.sql file import failed! ************************************************************************* Error executing: call insert_public_group_in_x_group_table(); com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_group`, CONSTRAINT `x_group_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`)) SQLException : SQL state: 23000 com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_group`, CONSTRAINT `x_group_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452 2018-10-25 02:13:39,667[JISQL] /usr/jdk64/jdk1.8.0_112/bin/java-cp /usr/hdp/current/ranger-admin/ews/lib/mysql.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://prd-rangervip01.xxxx.nm1/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = '006' and active = 'Y';" 2018-10-25 02:13:40,120[JISQL] /usr/jdk64/jdk1.8.0_112/bin/java-cp /usr/hdp/current/ranger-admin/ews/lib/mysql.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://prd-rangervip01.xxxx.nm1/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='006' and active='N' and updated_by='prd-xxx115.xxx.nm1';" 2018-10-25 02:13:40,526[E] 006-createdefaultpublicgroup.sql import failed! ************************************************************************* Error executing: INSERT INTO x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status) VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) SQLException : SQL state: 23000 com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452 2018-10-25 02:11:24,781[E] ranger_core_db_mysql.sql file import failed! ************************************************************************* Error executing: call insert_public_group_in_x_group_table(); com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_group`, CONSTRAINT `x_group_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`)) SQLException : SQL state: 23000 com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`ranger`.`x_group`, CONSTRAINT `x_group_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452 2018-10-25 02:13:39,667[JISQL] /usr/jdk64/jdk1.8.0_112/bin/java-cp /usr/hdp/current/ranger-admin/ews/lib/mysql.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://prd-rangervip01.xxxx.nm1/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = '006' and active = 'Y';" 2018-10-25 02:13:40,120[JISQL] /usr/jdk64/jdk1.8.0_112/bin/java-cp /usr/hdp/current/ranger-admin/ews/lib/mysql.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://pprd-rangervip01.xxxx.nm1/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='006' and active='N' and updated_by='prd-xxx115.xxx.nm1';" 2018-10-25 02:13:40,526[E] 006-createdefaultpublicgroup.sql import failed! I can see that few .sql imports are FAILED here. I have also followed the correct steps to be executed in MySQL: Referred URLs: 1. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_security/content/configuring_mysql_for_ranger.html 2. https://community.hortonworks.com/questions/214821/cant-install-ranger.html Please provide any technical suggestions to resolve this issue. Thanks, Shesh Kumar
... View more
Labels:
08-10-2018
10:09 AM
@Sandeep Nemuri Does it disable Hadoop's "chmod" command? User should be able to run command like this -- hadoop fs -ls / but not this -- hadoop fs -chmod 777 /hdfs/path Thanks, Shesh
... View more
- « Previous
-
- 1
- 2
- Next »