About techongig

techongig · ‎05-10-2017

What is the purpose of spnego.service.keytab in a secure hadoop cluster

techongig · ‎05-10-2017

@amankumbare thanks for the detailed explanation

techongig · ‎05-02-2017

hey @mqureshi thanks for providing the information. Apart from other zookeeper uses, so I assume the communication flow is to be like Producer will contact the zookeeper to get broker address and start producing messages to the brokers. Consumer will poll the brokers for messages and update the consumed message offset to Zookeeper.

techongig · ‎05-02-2017

Am assuming, users are not authorized to create topics directly (except kafka user) and topics can only be created while producing if there is non-existent topic provided user is authorized on ranger kafka policy. How can users (apart from kafka user) create topics with partitions and replication factor on a kerberized ranger enabled cluster? Am able to create topic while producing but below errors is shown up and am interested in creating topics with partition and replication /usr/hdp/current/kafka-brokers/bin/kafka-console-producer.sh --broker-list hadoop1.test.local:6667 --topic mytest5 --security-protocol PLAINTEXTSASL WARN Error while fetching metadata [{TopicMetadata for topic test5 -> No partition metadata for topic test5 due to kafka.common.LeaderNotAvailableException}] for topic [test5]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

techongig · ‎04-30-2017

Thanks slachterman

techongig · ‎04-30-2017

Thank you for the simple and crisp explanation

techongig · ‎04-30-2017

In Kafak, what is the difference while producing with single broker host and multiple broker host ? For example, Single Broker Hosts # /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list hadoop1.test.com:6667 --topic testtopic --security-protocol PLAINTEXTSASL For example, Multiple Broker Hosts # /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list hadoop1.test.com:6667,hadoop2.test.com:6667 --topic testtopic --security-protocol PLAINTEXTSASL

techongig · ‎04-30-2017

What is the role of zookeeper here with Kafka apart from distributed co-ordination? Are Kafka Topics created under zookeeper when cluster is kerberized and Ranger is enabled for authorization? I know the messages are stored under logs.dir path. Does zookeeper store metadata of topics etc? Does zookeeper store topics only when native ACL is used and not in Ranger?

techongig · ‎04-30-2017

Can you please provide the commands on how to login to zookeeper nodes and to delete the topics. Am also stuck at the step which says "topic is marked for deletion".

techongig · ‎04-29-2017

What are the way we can access the hive from remote server. What are differences between hive thrift server and hive thrift client

techongig · ‎04-26-2017

Great. this was the answer I was looking for.

techongig · ‎04-25-2017

thanks mqureshi for the response. I have HDP 2.4 Cluster with Active & Standby Namenode. Standby Namenode is down from few days due to hardware issue. I do understand that once standby namenode is back up, it updates itself with the edits logs from the journal nodes. In addition to that I believe standby namenode also performs checkpoint operation (merging of fsimage and edits) and uploads it to Active Namenode based on the property dfs.namenode.checkpoint.period. (which will be done by Secondary Namenode in non HA environment). Now in this case when Standby namenode is down, does check pointing happen or not or any solution for that? Appreciate your help

techongig · ‎04-25-2017

When in case Standby Namenode is down and we have only Active Namenode currently up, who does the checkpoint operations of FSIMAGE and EDITS logs and can you please explain how it happens?

techongig · ‎02-23-2017

sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager securityManager.sessionManager = $sessionManager securityManager.sessionManager.globalSessionTimeout = 86400000 Please uncomment these lines in shiro.ini file and restart zeppelin.

Online	Offline
Last Visited	‎03-12-2019 06:39 AM

Date Registered	‎02-23-2017 12:29 PM
Last Visited	‎03-12-2019 06:39 AM
Total Messages Posted	14
Total Kudos Received	2

What is the purpose of spnego.service.keytab in a ...

Re: How to remove topics to be deleted when deleti...

Re: Are Kafka Topics created under zookeeper when ...

How can other users (apart from kafka user) create...

Re: In Kafak, what is the difference while produci...

Re: In Kafak, what is the difference while produci...

In Kafak, what is the difference while producing w...

Are Kafka Topics created under zookeeper when clus...

Re: How to remove topics to be deleted when deleti...

What are the ways of accessing Hive from remote se...

Re: When Standby Namenode is down, who does checkp...

Re: When Standby Namenode is down, who does checkp...

When Standby Namenode is down, who does checkpoint...

Re: Zeppelin UI Welcome page is blank