Member since
04-04-2018
80
Posts
32
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5691 | 10-28-2017 05:13 AM |
02-01-2019
05:22 AM
Hello, How can we create kafka topics directly into hdfs.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Kafka
02-01-2019
05:21 AM
Hello, How can we create kafka topics directly into hdfs.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Kafka
02-01-2019
05:15 AM
Hi Sandeep, Even we format namenode again we can loose the data. My question is why only namenode format why not datanodes.
... View more
12-15-2018
03:41 PM
Hello, Why does not datanodes formatted during formatting the namenode server. Thank you.
... View more
Labels:
- Labels:
-
Apache Hadoop
04-04-2018
04:27 AM
Hi, I will be grateful if someone can answer the following questions on Kafka. 1. What is inter.broker.protocol.version and what it does. 2. What is log.message.format.version and what is does 3. How log.message.format and inter.broker.protocol relate each other. 4. When and on what basis need to the change log.message.format and inter.broker.protocol version. 5. What changes are required on consumer and producer side after changed the version. Thanks, Nilesh
... View more
Labels:
- Labels:
-
Apache Kafka
04-02-2018
06:54 PM
@Geoffrey Shelton Okot Thank you for your reply. I have some following queries, 1. What is inter.broker.protocol.version and what it does. 2. How log.message.format and inter.broker.protocol relate each other. 3. Could you please explain in detail the use of log.message.format.version. 4. When and on what basis need to the change log.message.format and inter.broker.protocol version. 5. What changes are required on consumer and producer side after changed the version. Thanks, Nilesh
... View more
04-02-2018
10:40 AM
Hi, 1. What is log.message.format.version in Kafka 2. What is the use of log.message.format.version in Kafka 3.What could be the version of log.message.format Thanks, Nilesh
... View more
Labels:
- Labels:
-
Apache Kafka
01-25-2018
12:45 PM
What does inter.broker.protocol.version in Kafka?
... View more
- Tags:
- Kafka
Labels:
- Labels:
-
Apache Kafka
01-16-2018
01:06 PM
@Harald Berghoff Thank you for the reply. How to check this in zookeeper.
... View more
01-16-2018
01:00 PM
Team, I am receiving following an error while executing kafka mirror-maker command. Kerberos is configured on source and target cluster. kafka@hdpensrv$ /usr/hdp/current/kafka-broker/bin/kafka-mirror-maker.sh --consumer.config /tmp/sourceClusterConsumer.properties --producer.config /tmp/targetClusterProducer.properties --whitelist="topic_test" [2018-01-16 06:21:45,493] ERROR Exception when starting mirror maker. (kafka.tools.MirrorMaker$)
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188)
at kafka.tools.MirrorMaker$MirrorMakerProducer.<init>(MirrorMaker.scala:626)
at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:213)
at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:71)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277)
... 4 more
... View more
Labels:
- Labels:
-
Apache Kafka
11-30-2017
03:14 PM
Hi, How producer and consumer identify the leader in Kafka. Thank you.
... View more
- Tags:
- Kafka
Labels:
- Labels:
-
Apache Kafka
11-29-2017
03:36 PM
Hi, Is there any command to check Kafka version. Thank you.
... View more
Labels:
- Labels:
-
Apache Kafka
11-27-2017
09:21 AM
Hi @ Abdelkrim Hadjidj Thank you for the reply. My question is I am ingesting same data again and again on the same topic then what will happen? Will it overwrite the data or will make another copy of data or it will just ignore and notify? Thank you.
... View more
11-22-2017
02:43 PM
Team, What will be happened if we ingest same data, in particular, Kafka topic again? Will it overwrite the data or will make another copy of data or it will just ignore and notify.
... View more
Labels:
- Labels:
-
Apache Kafka
11-01-2017
09:54 AM
Hi, We are trying to integrate Nifi and Ranger with LDAP. Ranger is successfully integrated with LDAP and i am able to login with the AD user. But when we are trying to integrate LDAP with NIFI using Ranger we facing issues. Will elaborate it using two scenarios. 1) When we select user as "{USER}" than we can login to Nifi UI using AD users and it also gets logged in with Anonymous users. Please find below screenshot. capture1.png capture2.png 2) If we remove "{USER}" from select user and put AD user for eg "apatank". Than we am unable to login and it gives me "Insufficient Permissions" error. Please find below screenshot. capture3.png capture4.png How can we remove anonymous user "{USER}" and use AD users in select users option in Ranger Ploicy and login to NIFI UI? Please help us in resolving above issue. Regards, Avdhoot
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Ranger
10-28-2017
05:13 AM
Thank you for sharing the URL. I am looking for topic cloning within the same cluster. Can we do this.
... View more
10-27-2017
01:58 PM
Hi, I found following log files in /var/log/log_hadoop/log/kafka/audit/hdfs/spool/archive path and these are taking lot of space. What are these files? Can I delete these files? How to control it? 11G spool_kafka_20171022-0406.23.log 11G spool_kafka_20171023-0406.30.log 11G spool_kafka_20171024-0406.30.log 11G spool_kafka_20171025-0406.31.log 11G spool_kafka_20171026-0406.32.log
... View more
Labels:
- Labels:
-
Apache Kafka
10-27-2017
11:59 AM
Can we clone single Kafka topic? If yes then how?
... View more
- Tags:
- Kafka
Labels:
- Labels:
-
Apache Kafka
10-16-2017
01:17 PM
1 Kudo
Hi Experts, We have 3 nodes of Kafka broker cluster setup along with 3 nodes of the zookeeper. Following are some queries while creating topics in the cluster. I am running following command to create topic ./kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic portfolio_break_stat Here are my questions, 1. If I put replication-factor 1 then on which node it will create the topic. It will create on all nodes or only one node. If only one node how Kafka decide it? 2. If I did not mention replication-factor then how the topic is created on the node and which node it will pick up?
... View more
Labels:
- Labels:
-
Apache Kafka
04-17-2017
02:13 PM
Team, What is use of ambari-qa account. Why it is require in hdp cluster. Can It replace with "root" account during hdp installation via ambari.
... View more
Labels:
04-11-2017
10:05 AM
Team, How do I download Ambari-Server 2.5.0 and agent RPM file.
... View more
- Tags:
- ambari-server
Labels:
- Labels:
-
Apache Ambari
03-29-2017
05:51 AM
@Mike Riggs We have already having Kafka setup but no one knows flow of Kafka then in this case how do I identified from where Kafka taking the data and where it store in hadoop.
... View more
03-27-2017
03:17 PM
Dear experts, How to identify Kafka source and destination (Hdfs,Hbase) path
... View more
Labels:
- Labels:
-
Apache Kafka
09-08-2016
07:11 AM
1 Kudo
Dear Team, If i used linux based encryption on hadoop cluster then is it encrypt hdfs data.
... View more
Labels:
- Labels:
-
Apache Hadoop
04-27-2016
02:11 PM
@Ravi Mutyala Does sqoop import support orc file. Where is compressed parameter.
... View more
04-26-2016
02:37 PM
1 Kudo
Hi Team, How to import oracle tables in hive with compressed format via sqoop command. Where to configure compression. i.e MapReduce or HDFS or Hive. Should I configure compression for Mapreduce ,HDFS and Hive. Which is the best compression formats supported by Hive.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Sqoop