About Nilesh

Nilesh · ‎02-01-2019

Hello, How can we create kafka topics directly into hdfs.

Nilesh · ‎02-01-2019

Hello, How can we create kafka topics directly into hdfs.

Nilesh · ‎02-01-2019

Hi Sandeep, Even we format namenode again we can loose the data. My question is why only namenode format why not datanodes.

Nilesh · ‎12-15-2018

Hello, Why does not datanodes formatted during formatting the namenode server. Thank you.

Nilesh · ‎11-29-2018

Hi, I require kafka-graphite.jar and metrics-graphite.jar files compatible for Kafka version 1.0.0

Nilesh · ‎04-04-2018

Hi, I will be grateful if someone can answer the following questions on Kafka. 1. What is inter.broker.protocol.version and what it does. 2. What is log.message.format.version and what is does 3. How log.message.format and inter.broker.protocol relate each other. 4. When and on what basis need to the change log.message.format and inter.broker.protocol version. 5. What changes are required on consumer and producer side after changed the version. Thanks, Nilesh

Nilesh · ‎04-02-2018

@Geoffrey Shelton Okot Thank you for your reply. I have some following queries, 1. What is inter.broker.protocol.version and what it does. 2. How log.message.format and inter.broker.protocol relate each other. 3. Could you please explain in detail the use of log.message.format.version. 4. When and on what basis need to the change log.message.format and inter.broker.protocol version. 5. What changes are required on consumer and producer side after changed the version. Thanks, Nilesh

Nilesh · ‎04-02-2018

Hi, 1. What is log.message.format.version in Kafka 2. What is the use of log.message.format.version in Kafka 3.What could be the version of log.message.format Thanks, Nilesh

Nilesh · ‎01-28-2018

What is segment in Kafka. What it does?

Nilesh · ‎01-25-2018

@Sandeep Nemuri I tried this URL. This document is not updated

Nilesh · ‎01-25-2018

What does inter.broker.protocol.version in Kafka?

Nilesh · ‎01-16-2018

@Harald Berghoff Thank you for the reply. How to check this in zookeeper.

Nilesh · ‎01-16-2018

Team, I am receiving following an error while executing kafka mirror-maker command. Kerberos is configured on source and target cluster. kafka@hdpensrv$ /usr/hdp/current/kafka-broker/bin/kafka-mirror-maker.sh --consumer.config /tmp/sourceClusterConsumer.properties --producer.config /tmp/targetClusterProducer.properties --whitelist="topic_test" [2018-01-16 06:21:45,493] ERROR Exception when starting mirror maker. (kafka.tools.MirrorMaker$) org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188) at kafka.tools.MirrorMaker$MirrorMakerProducer.<init>(MirrorMaker.scala:626) at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:213) at kafka.tools.MirrorMaker.main(MirrorMaker.scala) Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:71) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277) ... 4 more

Nilesh · ‎11-30-2017

Hi, How producer and consumer identify the leader in Kafka. Thank you.

Nilesh · ‎11-29-2017

Hi, Is there any command to check Kafka version. Thank you.

Nilesh · ‎11-27-2017

Hi, If I do partition reassignment on the topic then what will happen internally? 1. The data which resided on existing broker will also move to new broker or not.OR 2. The only topic name will move to the new broker when data come to this topic will reside on the new broker.

Nilesh · ‎11-27-2017

Hi @ Abdelkrim Hadjidj Thank you for the reply. My question is I am ingesting same data again and again on the same topic then what will happen? Will it overwrite the data or will make another copy of data or it will just ignore and notify? Thank you.

Nilesh · ‎11-22-2017

Team, What will be happened if we ingest same data, in particular, Kafka topic again? Will it overwrite the data or will make another copy of data or it will just ignore and notify.

Nilesh · ‎11-01-2017

Hi, We are trying to integrate Nifi and Ranger with LDAP. Ranger is successfully integrated with LDAP and i am able to login with the AD user. But when we are trying to integrate LDAP with NIFI using Ranger we facing issues. Will elaborate it using two scenarios. 1) When we select user as "{USER}" than we can login to Nifi UI using AD users and it also gets logged in with Anonymous users. Please find below screenshot. capture1.png capture2.png 2) If we remove "{USER}" from select user and put AD user for eg "apatank". Than we am unable to login and it gives me "Insufficient Permissions" error. Please find below screenshot. capture3.png capture4.png How can we remove anonymous user "{USER}" and use AD users in select users option in Ranger Ploicy and login to NIFI UI? Please help us in resolving above issue. Regards, Avdhoot

Nilesh · ‎10-28-2017

Thank you for sharing the URL. I am looking for topic cloning within the same cluster. Can we do this.

Nilesh · ‎10-28-2017

Can we delete these files?

Nilesh · ‎10-27-2017

Hi, I found following log files in /var/log/log_hadoop/log/kafka/audit/hdfs/spool/archive path and these are taking lot of space. What are these files? Can I delete these files? How to control it? 11G spool_kafka_20171022-0406.23.log 11G spool_kafka_20171023-0406.30.log 11G spool_kafka_20171024-0406.30.log 11G spool_kafka_20171025-0406.31.log 11G spool_kafka_20171026-0406.32.log

Nilesh · ‎10-27-2017

Can we clone single Kafka topic? If yes then how?

Nilesh · ‎10-16-2017

Hi Experts, We have 3 nodes of Kafka broker cluster setup along with 3 nodes of the zookeeper. Following are some queries while creating topics in the cluster. I am running following command to create topic ./kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic portfolio_break_stat Here are my questions, 1. If I put replication-factor 1 then on which node it will create the topic. It will create on all nodes or only one node. If only one node how Kafka decide it? 2. If I did not mention replication-factor then how the topic is created on the node and which node it will pick up?

Nilesh · ‎04-17-2017

Team, What is use of ambari-qa account. Why it is require in hdp cluster. Can It replace with "root" account during hdp installation via ambari.

Nilesh · ‎04-13-2017

Hi Experts, Is it possible to install hdp 2.5.0 via domain account. ie. During installation can I submit "SSH user account" to domain name and proceed for installation. OR It require only "root" account OR It require normal local user account created on linux all servers.

Nilesh · ‎04-12-2017

Dear Team, Following an error received during hdp installation. Installation is going with domain1\user id using /usr/lib/python2.6/site-packages/ambari_server cluster primary OS: redhat6 with user 'DOMAIN1\user'with ssh Port '22' sshKey File /var/run/ambari-server/bootstrap/4/sshKey password File null using tmp dir /var/run/ambari-server/bootstrap/4 ambari: cshdpdevdb8003.example.com; server_port: 8080; ambari version: 2.5.0.3; user_run_as: root INFO:root:Executing parallel bootstrap. Bootstrap process timed out. It was destroyed.

Nilesh · ‎04-11-2017

Thanks Jay :)

Nilesh · ‎04-11-2017

Team, How do I download Ambari-Server 2.5.0 and agent RPM file.

Nilesh · ‎03-29-2017

@Mike Riggs We have already having Kafka setup but no one knows flow of Kafka then in this case how do I identified from where Kafka taking the data and where it store in hadoop.

Online	Offline
Last Visited	‎04-12-2018 06:26 AM

Date Registered	‎04-04-2018 01:30 AM
Last Visited	‎04-12-2018 06:26 AM
Total Messages Posted	80
Total Kudos Received	31

Re: How to Kafka topic clone.

How to create Kafka topics directly on HDFS

How to create Kafka topics directly on HDFS

Re: Why not datanode format

Why not datanode format

Jar required for Kafka 1.0.0 - Graphite monitoring

Kafka Basics

Re: log.message.format.version in Kafka

log.message.format.version in Kafka

Kafka Segment

Re: Kafka | MirrorMaker | Kerberos

inter.broker.protocol.version in kafka

Re: How producer and consumer identify the leader ...

Kafka | MirrorMaker | Kerberos

How producer and consumer identify the leader in K...

How to check Kafka version

Kafka partition reassignment

Re: Ingest same data in Kafka topic

Ingest same data in Kafka topic

Insufficient Permissions error while login to Nifi...

Re: How to Kafka topic clone.

Re: Kafka spool log files

Kafka spool log files

How to Kafka topic clone.

Kafka topic | created on which node

Ambari-qa user || Ambari Installation

HDP 2.5.0 Installation | User Account

HDP 2.5 installation error

Re: Ambari-Server and Agent RPM Download

Ambari-Server and Agent RPM Download

Re: Kafka | Source and Destination Path