Member since
01-19-2017
3679
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 924 | 06-04-2025 11:36 PM | |
| 1527 | 03-23-2025 05:23 AM | |
| 756 | 03-17-2025 10:18 AM | |
| 2721 | 03-05-2025 01:34 PM | |
| 1802 | 03-03-2025 01:09 PM |
01-05-2019
10:39 PM
@huzaira bashir Please find a complete process of the kerberization process
... View more
01-04-2019
03:12 PM
1 Kudo
@harish Yes for sure, that's doable I am assuming you have set up 2 kdc's on different networks but accessible to the cluster, Assumptions: You MUST have successfully configure the 2 master and slave KDC's my realm =REALM
Master host=master-kdc.test.com
Slave host=slave-kdc.test.com Contents of /var/kerberos/krb5kdc/kpropd.acl: host/master-kdc.test.com@REALM
host/slave-kdc.test.com@REALM # Create the configuration for kpropd on both the Master and Slave KDC hosts: # Create /etc/xinetd.d/krb5_prop with the following contents. service krb_prop
{
disable = no
socket_type = stream
protocol = tcp
user = root
wait = no
server = /usr/sbin/kpropd
} # Configure xinetd to run as a persistent service on both the Master and Slave KDC hosts: # systemctl enable xinetd.service
# systemctl start xinetd.service # Copy the following files from the Master KDC host to the Slave KDC host: /etc/krb5.conf
/var/kerberos/krb5kdc/kadm5.acl
/var/kerberos/krb5kdc/kdc.conf
/var/kerberos/krb5kdc/kpropd.acl
/var/kerberos/krb5kdc/.k5.REALM # Perform the initial KDC database propagation to the Slave KDC: # kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans
# kprop -f /usr/local/var/krb5kdc/slave_datatrans slave-kdc.REALM # Start the Slave KDC : # systemctl enable krb5kdc
# systemctl start krb5kdc # Script to propagate the updates from the Master KDC to the Slave KDC. Create a cron job, or the like, to run this script on a frequent basis. #!/bin/sh
#/var/kerberos/kdc-slave-propogate.sh
kdclist = "slave-kdc.customer.com"
/sbin/kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans
for kdc in $kdclist
do
/sbin/kprop -f /usr/local/var/krb5kdc/slave_datatrans $kdc
done How to test the KDC HA is to shut down the master KDC as start the slave KDC note both KDC's should NEVER be running at the same time, the crontab script should do the propagation of all changes in the KDC database in the master to the slave. CAUTION Run the kprop before shutting down the master KDC then to test the kdc HA log on to the cluster linux CLI follow the below steps my steps I am using the root user Switch user to hive/spark/Yarn etc # su - hive Check if the hive user still has valid Kerberos ticket The below output shows the hive user still has a valid ticket $ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: hdfs-host1@{REALM}
Valid starting Expires Service principal
12/28/16 22:57:11 12/29/16 22:57:11 krbtgt/{REALM}@{REALM} renew until 12/28/16 22:57:11
12/28/16 22:57:11 12/29/16 22:57:11 HTTP/host1.test.com@{REALM} renew until 12/28/16 22:57:11
12/28/16 22:57:11 12/29/16 22:57:11 HTTP/host1.com@{REALM} renew until 12/28/16 22:57:11 # Destroy the Kerberos tickets as user hive $ kdestroy Running the previous command shouldn't give you any lines, now try getting a valid ticket by running the following command format {kinit -kt $keytab $principal} $ kinit -kt /etc/security/keytabs/hive.keytab {PRINCIPAL} Repeating the klist should give the hive user a valid ticket this will validate that the HA is functioning well.
... View more
01-04-2019
02:15 PM
@huzaira bashir At least I am reassured about the previous screenshot. From the screenshot, I don't see domain which should be in the format and comma separated if your REALM is TEST.COM note the dot(.) .test.com,test.com And the Kadmin too, meanwhile can you share a tokenized version of you krb5.conf,kdc.conf and kadm5.acl most important ensure these 2 daemons are running Enable auto start # systemctl enable krb5kdc
# systemctl enable kadmin Start the daemons # /etc/rc.d/init.d/krb5kdc start
# /etc/rc.d/init.d/kadmin start or # systemctl start krb5kdc
# systemctl start kadmin Whichever is applicable HTH
... View more
01-03-2019
05:00 PM
@huzaira bashir What is the HDP version? The screenshot doesn't look a typical MIT Kerberos enabling UI? Could you be using the AD as KDC? Having said that can you share how you procedure used? Can you share the Kerberos enabling screenshots from Ambari? If you could answer promptly with the above info then it would help a great deal. HTH
... View more
01-03-2019
02:05 PM
@Michael Bronson Nodemanager is a slave process of YARN so you should drill down the YARN, in my case I just intentionally brought down my node manager so the problematic Nodemanager should show. Go to the ResourceManager UI check the nodes link on the left side of the screen. All your NodeManagers should be listed there and the reason for it being listed as unhealthy may be shown here. It is most likely due to yarn local dirs or log dirs. You may be hitting the disk threshold for this. Finally checks the logs look in /var/log/hadoop-yarn/yarn and NOT in /var/log/hadoop/yarn
... View more
01-03-2019
10:10 AM
@Vinay So now the interactive query is running fine and it no longer throws errors, except that you can't see the other databases except the "DEFAULT" ? IN HDP 3.0 spark uses its own separate catalog this should explain why can't see any hive databases. ToYou should use the HiveWarehouseConnector. work with hive databases please follow this documentation Configuring hiveWarehouseConnector Please revert HTH
... View more
01-02-2019
11:49 AM
@Vinay Yes, you need to enable Interactive query. Did you follow these steps LLAP & Interactive query Remember also to enable YARN pre-emption via YARN config HTH
... View more
01-01-2019
10:38 PM
@john y Can you use http:// localhost:8080 instead of www !
... View more
01-01-2019
05:13 PM
@max mouse There isn’t a one-and-only tool that can do everything equally well and address all of your requirements. Combining tools that do different things in better ways allows for a buildup in functionality and increased flexibility in handling a larger set of scenarios. Depending on your needs, both NiFi and Flume can act as Kafka producers and/or consumers. HTH
... View more
01-01-2019
03:43 PM
@Michael Bronson This is what I was trying to say, below is my single node HDP and AD server. In your case, unless you have a DNS server you should have 3 entries for master1,master2 and master3
... View more