Member since
01-19-2017
3676
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 609 | 06-04-2025 11:36 PM | |
| 1175 | 03-23-2025 05:23 AM | |
| 579 | 03-17-2025 10:18 AM | |
| 2183 | 03-05-2025 01:34 PM | |
| 1373 | 03-03-2025 01:09 PM |
09-07-2017
06:42 PM
@Sam Red For a user to submit successfully or even connect to hive in a kerberized environment the user MUST first get a valid Kerberos ticket. Can you users list the principals attached to their keytabs eg for user opera1 # su - opera1$
$ klist
klist: No credentials cache found (filename: /tmp/krb5cc_1001) Above output show no valid kerberos ticket ,then follow the below steps as user opera1 assuming your keytabs are in that directory and the REALM is DEV.COM $ klist -kt /etc/security/keytabs/opera1.keytab
Keytab name: FILE:/etc/security/keytabs/opera1.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
1 08/24/2017 18:28:26 opera1@DEV.COM
$ kinit -kt /etc/security/keytabs/opera1.keytab opera1@UGANDA.COM
$ hive The user should be able now to run a job on the cluster ! Please let me know
... View more
09-06-2017
10:05 PM
@Vicente Ciampa Please find below some quick answers to your questions 1) there is specific documentation to configure the services: HDFS, YARN, MapReduce2, Hive, HBase, Zookeeper, Ambari Infra, Ambari Metrics, Atlas, Kafka, Knox, Ranger, Spark2 and Zeppelin Notebook for KERBEROS There is no specific action to take if your kerberization is done through the Ambari Wizard, as it takes care of generating the principals and corresponding keytabs. 2) The task is to install kerberos so that users of the active directory can access (there is some documentation to help me the best practices to do this) Yes have a look at this site_one site_two Also see this video without audio 3) After reinstalling I will send you the files you asked for so please support me, thanks I will wait for the files requested. 4) You could connect by teamviewer so you can observe better. What do you think? Yes I can help by team viewer provided it's not a production environment otherwise you need the Hortoworks support team. Hope that answers your questions
... View more
09-06-2017
09:46 PM
@forest lin There are a couple of things that might have gone wrong. How many nodes in your cluster?
Have you successfully enabled it through the Ambari wizard?
Did you copy the krb5.conf from the KC to all hosts?
Make sure your KDC is up and running ensure autostart on OS reboot is enabled. I noticed you used REALM in lowercase 🙂 kadmin -p admin/admin@dev.com it should be kadmin -p admin/admin@DEV.COM Can you copy and paste the contents following config files. Remember to mask sensitive info ONLY krb5.conf
kadm5.acl
kdc.conf Can you log on to the KDC using see below or sudo option where applicable # kadmin.local
# sudo kadmin.local
Authenticating as principal admin/admin@DEV.COM with password.
kadmin.local: addprinc test3@DEV.COM
WARNING: no policy specified for test3@DEV.COM; defaulting to no policy
Enter password for principal "test3@DEV.COM":
Re-enter password for principal "test3@DEV.COM": {Master_KDC_password}
Principal "test3@DEV.COM" created Type q to quit kadmin. Create a keytab in the current directory for the test3 user (use appropriate command) # sudo ktutil
# ktutil ktutil: addent -password -p test3@DEV.COM -k 1 -e RC4-HMAC
Password for test3@DEV.COM:
ktutil: wkt test3.keytab
ktutil: q Copy the keytab to test3 user home directory and change the permissions # chown test3:test3 test3.keytab Test the validity of the keytab against the principal created earlier on # klist -kt /etc/security/keytabs/test3.keytab
Keytab name: FILE:/etc/security/keytabs/test3.keytab
KVNO Timestamp Principal
-----------------------------------------------------------------------------
1 07/18/2017 10:46:27 test3@DEV.COM Grab a Kerberos ticket note the concatenation of keytab/principal # kinit -kt /etc/security/keytabs/test3.keytab test3@DEV.COM Check if the above command gives a valid Kerberos ticket # klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: test3@DEV.COM
Valid starting Expires Service principal
07/18/2017 10:53:48 07/19/2017 10:53:48 krbtgt/DEV.COM@DEV.COM All is set if need be repeat the above for all users you intend to create Please revert
... View more
09-06-2017
08:34 PM
@Sam Red Assumption KDC is running KDC is created KDC user and master password is available REALM TEST.COM Edgenode : for users user is root or sudoer To succeed you MUST give users access to your home directory 🙂 Not a good solution security wise ....If these are unix users then create the keytabs in e.g /tmp and later copy them to the respective home directories and make sure to change the correct permissions on the keytabs. You will notice a node dedicated to users EDGE NODE, all client softwares are installed here and not on the data or name nodes! # cd /tmp
# sudo kadmin.local
Authenticating as principal root/admin@TEST.COM with password.
kadmin.local: addprinc user1@TEST.COM
WARNING: no policy specified for user1@TEST.COM; defaulting to no policy
Enter password for principal "user1@TEST.COM":
Re-enter password for principal "user1@TEST.COM":
Principal "user1@TEST.COM" created. -----do the same for all other user too ------ addprinc user2@TEST.COM
addprinc user3@TEST.COM
addprinc usern@TEST.COM The keytabs with be generated in the current directory # generate keytab for user1
##########################################
# sudo ktutil
ktutil: addent -password -p user1@TEST.COM -k 1 -e RC4-HMAC
Password for user1@TEST.COM:
ktutil: wkt user1.keytab
ktutil: q You MUST repeat the above for all your users # chown user1:user1 user1.keytab Again do the above for all users and copy the keytabs from the kdc to edgenode, change the ownership of the respective keytabs # chown user1:user1 user1.keytab Validate the principals in this example the keytabs are in /etc/security/keytabs # klist -kt /etc/security/keytabs/user1.keytab
Keytab name: FILE:/etc/security/keytabs/user1.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
1 07/18/2017 10:46:27 user1@TEST.COM Test the new user1 should try grabbing a kerberos ticket # kinit -kt /etc/security/keytabs/user1.keytab user1@TEST.COM The below command should show athe validity of the kerbero ticket # klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: user1@TEST.COM
Valid starting Expires Service principal
06/09/2017 10:53:48 06/10/2017 10:53:48 krbtgt/TEST.COM@TEST.COM You should be okay now
... View more
09-06-2017
07:43 PM
1 Kudo
@Mohammed Ayaz HDP 2.3 Ambari 2.1.2 Non kerberized. Installed HortonworksHiveODBC32 and HortonworksHiveODBC64 bit Laptop C:\Windows\System32\drivers\etc\hosts entry # localhost name resolution is handled within DNS itself.
#127.0.0.1 localhost
#::1 localhost
192.168.0.80 gulu.uganda.com gulu
192.168.0.157 bombo.uganda.com bombo
192.168.0.213 sandbox.hortonworks.com sandbox Logged to my I node cluster using putty [hive@bombo ~]$ hive WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in file:/etc/hive/2.3.6.0-3796/0/hive-log4j.properties
hive> show databases;
OK
default Time taken: 4.796 seconds, Fetched: 1 row(s)
hive> SET hive.cli.print.header=true;
hive> create database amin;
OK
Time taken: 1.79 seconds
hive> use amin;
OK
Time taken: 0.355 seconds Created a table hive>CREATE TABLE amin.orders_hive
(
Order_ID string
, Customer_ID int
, Store_ID int
-- , Order_Datetime_Orig timestamp
, Order_Datetime timestamp
, Ship_Completion_Datetime timestamp
, Return_Datetime timestamp
, Refund_Datetime timestamp
, Payment_Method_Code string
)
-- PARTITIONED BY (Order_Datetime timestamp)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
LOCATION '/apps/hive/warehouse/'; I choose 2 types of connection with and without zookeeper and my connection was successful. I then created a third connection with my user/password and that too was successful using option. See attached screenshots Hope that helps
... View more
09-06-2017
05:57 PM
@ Sam Red The error usually means that the port you are trying to open is being already used by another application try using netstat to see which ports are open and then use an available port. # netstat -nlp|grep 9995 Then look for the zeppelin port look for the PID second from left after java eg 19782 # lsof -n -i :9995 | grep LISTEN
java 19782 zeppelin 187u IPv6 294077 0t0 TCP *:palace-4 (LISTEN) Validate the PID # fuser -v -n tcp 9995 Look for the PID, then kill the process # kill -9 {PID} Now restart zeppelin it should bind
... View more
09-05-2017
06:10 PM
@btandel Good to know you got that message. Did you "Create Table As Select (CTAS) " from 2 different tables to see the lineage?
... View more
09-05-2017
03:17 PM
@Gjin 733 Good to know and happy ...... everything is working fine. I am not very conversant with Docker but Kerberos and your error were pointing to the krb5.conf. If you feel my response helped you resolve the issue then you can accept it to reward me. I should build some kafka docker on my cluster and test ..
... View more
09-05-2017
03:08 PM
@btandel Did you get a message like this at the end of the import? Hive Data Model imported successfully!!! Can you create any table using CTAS from an existing table as see the lineage in atlas? The following hive operations are captured by hive hook currently
create database create table/view, create table as select load, import, export DMLs (insert) alter database alter table (skewed table information, stored as, protection is not supported) alter view
... View more