@Geoffrey Shelton Okot Thank you for your reply. I have some following queries, 1. What is inter.broker.protocol.version and what it does. 2. How log.message.format and inter.broker.protocol relate each other. 3. Could you please explain in detail the use of log.message.format.version. 4. When and on what basis need to the change log.message.format and inter.broker.protocol version. 5. What changes are required on consumer and producer side after changed the version. Thanks, Nilesh ... View more

It allows you to run brokers of different versions in one cluster, to avoid a downtime of the cluster during the upgrade. Before you upgrade any broker, you set the inter.broker.protocol.version to the existing version on all brokers. Then you start upgrading broker by broker, the newer broker will still use the 'old' protocol to communicate with the other brokers. This keeps the cluster functional during the time when just some brokers are updated. Once all brokers are upgraded, you change the inter.broker.protocol.version to the new version and restart them one by one. More details here: https://kafka.apache.org/documentation/#upgrade ... View more

export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=<path to jaas.conf>" Try exporting the env variable before running the mirror maker. ... View more

Since Kafka version 0.9, Producers and consumers fetch all metadata details from Kafka brokers directly and Kafka brokers are in sync with Zookeeper. Each Kafka brokers have information about leader. ... View more

Just look at Kafka as a distibuted log file. even if you insert the same data, again and again, it just appends the data into a distibuted log file. ... View more

@Nilesh This document will walk you through, you should be able to access Nifi using yoour AD account ... View more

Yes, MirrorMaker is not putting a limitation on remote vs local cluster. It is designed for remote clusters because there is almost no need to do it locally. If you are mirroring a topic locally, you must rename it, and if you are going to rename it, then you have consumers/producers using data in both topics? You are replicating data within the same cluster for little gain while your consumers/producers can easily be configured to use the correct topic(s). ... View more

@Nilesh "ambari-qa" is Smoke Test user which performs smoke tests against cluster services as part of the install process. It also can perform these on-demand from the Ambari Web GUI. Example: When you run the service checks then you will find that this user acccount is actually is used to perform those checks. "Ambari UI" --> "Yarn" --> "Service Action" --> "Run Service Check" call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://erie2.example.com:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpcDRYnw 2>/tmp/tmp3Oclia''] {'logoutput': None, 'quiet': False} . ... View more

Thanks Jay :) ... View more