Member since
02-23-2017
14
Posts
2
Kudos Received
0
Solutions
05-10-2017
07:44 AM
What is the purpose of spnego.service.keytab in a secure hadoop cluster
... View more
Labels:
- Labels:
-
Apache Hadoop
05-10-2017
06:58 AM
@amankumbare thanks for the detailed explanation
... View more
05-02-2017
11:19 AM
hey @mqureshi thanks for providing the information. Apart from other zookeeper uses, so I assume the communication flow is to be like Producer will contact the zookeeper to get broker address and start producing messages to the brokers. Consumer will poll the brokers for messages and update the consumed message offset to Zookeeper.
... View more
05-02-2017
06:48 AM
2 Kudos
Am assuming, users are not authorized to create topics directly (except kafka user) and topics can only be created while producing if there is non-existent topic provided user is authorized on ranger kafka policy. How can users (apart from kafka user) create topics with partitions and replication factor on a kerberized ranger enabled cluster? Am able to create topic while producing but below errors is shown up and am interested in creating topics with partition and replication /usr/hdp/current/kafka-brokers/bin/kafka-console-producer.sh --broker-list hadoop1.test.local:6667 --topic mytest5 --security-protocol PLAINTEXTSASL
WARN Error while fetching metadata [{TopicMetadata for topic test5 ->
No partition metadata for topic test5 due to kafka.common.LeaderNotAvailableException}] for topic [test5]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Ranger
04-30-2017
05:48 PM
Thanks slachterman
... View more
04-30-2017
05:45 PM
Thank you for the simple and crisp explanation
... View more
04-30-2017
10:18 AM
In Kafak, what is the difference while producing with single broker host and multiple broker host ?
For example, Single Broker Hosts
# /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list hadoop1.test.com:6667 --topic testtopic --security-protocol PLAINTEXTSASL For example, Multiple Broker Hosts
# /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list hadoop1.test.com:6667,hadoop2.test.com:6667 --topic testtopic --security-protocol PLAINTEXTSASL
... View more
Labels:
- Labels:
-
Apache Kafka
04-30-2017
09:29 AM
What is the role of zookeeper here with Kafka apart from distributed co-ordination? Are Kafka Topics created under zookeeper when cluster is kerberized and Ranger is enabled for authorization? I know the messages are stored under logs.dir path. Does zookeeper store metadata of topics etc? Does zookeeper store topics only when native ACL is used and not in Ranger?
... View more
Labels:
- Labels:
-
Apache Kafka
04-30-2017
09:26 AM
Can you please provide the commands on how to login to zookeeper nodes and to delete the topics. Am also stuck at the step which says "topic is marked for deletion".
... View more
04-29-2017
10:50 AM
What are the way we can access the hive from remote server. What are differences between hive thrift server and hive thrift client
... View more
Labels:
- Labels:
-
Apache Hive
04-26-2017
07:58 PM
Great. this was the answer I was looking for.
... View more
04-25-2017
03:28 PM
thanks mqureshi for the response. I have HDP 2.4 Cluster with Active & Standby Namenode. Standby Namenode is down from few days due to hardware issue. I do understand that once standby namenode is back up, it updates itself with the edits logs from the journal nodes. In addition to that I believe standby namenode also performs checkpoint operation (merging of fsimage and edits) and uploads it to Active Namenode based on the property dfs.namenode.checkpoint.period. (which will be done by Secondary Namenode in non HA environment). Now in this case when Standby namenode is down, does check pointing happen or not or any solution for that? Appreciate your help
... View more
04-25-2017
02:15 AM
When in case Standby Namenode is down and we have only Active Namenode currently up, who does the checkpoint operations of FSIMAGE and EDITS logs and can you please explain how it happens?
... View more
Labels:
- Labels:
-
Apache Hadoop
02-23-2017
12:46 PM
sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
securityManager.sessionManager = $sessionManager
securityManager.sessionManager.globalSessionTimeout = 86400000 Please uncomment these lines in shiro.ini file and restart zeppelin.
... View more