Member since
01-19-2017
3679
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1554 | 06-04-2025 11:36 PM | |
| 2024 | 03-23-2025 05:23 AM | |
| 956 | 03-17-2025 10:18 AM | |
| 3614 | 03-05-2025 01:34 PM | |
| 2509 | 03-03-2025 01:09 PM |
01-24-2019
08:16 AM
@Lokesh Mukku In a Namenode HA there is no notion of IP address but Nameservice ID (geeklab)as illustrated in the screenshot. In the background, geeklab will be doing something like load balancing between (nn01,nn02) whichever is the Active at a particular time, for example, if nn01 is the active NN and all over a sudden it encounters an issue and goes down, the ZKFailoverController will detect that failure through the heartbeat misses and transparently failover to nn02. The Nameservice ID is used to avoid hard-coding the IP's in case you change the host during an upgrade or system failure. I gave you the example of DNS to explain it further..www.teamworks.com is a DNS entry used to resolve hostnames to a specific IP of the teamworks.com web server to machine-readable IP addresses (e.g. 192.168.106.81) behind the scenes www.teamworks.com to guard against any failure could have a couple of web servers (HA setup ) say 3 web servers serving content for www.teamworks.com with a load balancer (see screenshot LB.png) which act like the Nameservice ID in case a namenode/web server goes down the cluster/website will still be available as Nameservice ID will automatically failover point to the active name node controlled by ZKFailoverController . HTH
... View more
01-23-2019
06:02 PM
1 Kudo
@Lokesh Mukku The sequence of events you described is correct. The failover should be transparently handled by the ZKFailoverController . HA clusters use the nameservice ID (dfs.nameservices in hdfs-site.xml) to identify a single HDFS instance that may consist of multiple HA NameNodes. The nameservice ID acts like a DNS for further reading see this HW document HTH
... View more
01-23-2019
04:55 PM
@Ruslan Fialkovsky I think the topics created before changing the ACL's don't inherit the permissions. From the znode entry, I see you have one topic called test. To validate can you create a new topic and ensure you change them in your Nifi flow files and retest! Normally if you run rmr in zookeeper CLI the entry deleted should be created but I am not sure for Kafka topics. Please revert
... View more
01-23-2019
10:07 AM
@ Ruslan Fialkovsky Logon to Zk on DEV and PROD Check whether you have child znodes under ls /config/topics Validate that you have Kerberos entries in /etc/nifi/conf/nifi_jaas.conf example NiFiClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true keyTab="/etc/security/keytabs/nifi.service.keytab"
storeKey=true
useTicketCache=false
principal="nifi/{kdc_host}@REALM";
};
RegistryClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true keyTab="/etc/security/keytabs/nifi.service.keytab"
storeKey=true
useTicketCache=false
principal="nifi/{kdc_host}@REALM"; Please revert
... View more
01-23-2019
08:19 AM
@ Ruslan Fialkovsky Can you compare these files on both cluster, if it doesn't exist on the DEV then create them with correct owner and privileges Kafka /etc/kafka/conf/kafka_client_jaas.conf
/etc/kafka/conf/kafka_jaas.conf Zookeeper /etc/zookeeper/conf/zookeeper_jaas.conf
/etc/zookeeper/conf/zookeeper_client_jaas.conf After updating these files restart the zk and Kafka and retry HTH
... View more
01-23-2019
07:06 AM
@ Ruslan Fialkovsky Now you see that on DEV world: everyone has cdrwa while for kafka there is no entry so you will need to setAcl for kafka setAcl /config/topics world:anyone:cdrwa,sasl:kafka:cdrwa Did you grasp how to use the using superDigest to become a Zookeeper superuser, that's like the zookeeper root user who can add/change the ACL's ,let me know if you need help on that.
... View more
01-22-2019
07:51 PM
1 Kudo
@ Ruslan Fialkovsky Can you log on both cluster as zookeeper run kinit then $ /usr/hdp/current/zookeeper-server/bin/zkCli.sh
[zk: localhost:2181(CONNECTED) 0] getAcl /config/topics Compare the outputs there should be a difference between the 2 cluster sample output [zk: localhost:2181(CONNECTED) 1] getAcl /config/topics
'world,'anyone
: r
'sasl,'kafka
: cdrwa And on the other [zk: localhost:2181(CONNECTED) 1] getAcl /config/topics
'world,'anyone
: cdrwa If that's the case have a look at how to set the correct ACL for the /config/topics in this HCC document Please let me know
... View more
01-22-2019
05:36 PM
@ Ruslan Fialkovsky Are you doing some lab, this looks familiar can you share the link, I want to see if I can reproduce it on my cluster?
... View more
01-22-2019
03:18 PM
@ Ruslan Fialkovsky Is your nifi part of the kerberized cluster or is a separate HDF cluster? What are versions of the components NIF,HDP etc ?
... View more
01-22-2019
10:55 AM
1 Kudo
@Jeremy Jean-Jean There is no sense in installing zeppelin on all the nodes, Do you have YARN Client installed on the data nodes? Then submit using spark-submit --class <clasname> --master yarn --deploy-mode cluster <jars> <args> HTH
... View more