@sgowda Yes its possible. You can go to ambari kafka config page and add PLAINTEXTSASL://localhost:6667, PLAINTEXT://localhost:6668. Make sure the ports are unique. In PLAINTEXT connections user's identity set to Anonymous. You need to make sure you set the right ACLs using kafka-acls.sh command to give permissions to Anonymous user. ... View more

@Naresh Ar HDP started shipping Kafka from 0.8.1 (HDP-2.2) and HDP-2.4 we shipped kafka 0.9.0.1 . Kafka Streams became GA in apache kafka 0.10.0 release and it will be part of upcoming HDP-2.5 release. You don't need any other kafka streams packages you can access them from here http://repo.hortonworks.com/content/groups/public/org/apache/kafka/kafka-streams/ . As you can these are 0.9.0.1 release artifacts I advise you to wait for HDP-2.5 release which will contain most stable streams binaries. ... View more

@Régis LE BRETONNIC there is no such property as offsets.storage on kafka broker side i.e server.properties. offsets.storage is consumer side config. If you are using new consumer api than it defaults to storing the offsets in kafka topic but if you are using old high-level consumer api than it defaults to using zookeeper as storage. To change that make sure you add offsets.storage=kafka in consumer.properties. ... View more

@james.jones the metric you are looking is for old high level consumer api. For the new consumer api can you use kafka.consumer:type=consumer-fetch-manager-metrics,client-id=([-.w]+) and name=records-lag-max ... View more

did you try configuring external hostnames of ec2 boxes in advertised.listeners config. ... View more

We are releasing it in upcoming HDP-2.4 maint release. ... View more

@Yuri Chemolosov @Artem Ervits one of the workarounds I think of is to make storm-0.11-SNAPSHOT as dependency at the client side for the topology and use the StormSubmitter for 0.11-SNAPSHOT version. add the following repository in your pom.xml https://repository.apache.org/content/repositories/snapshots/ to get the latest snapshot for storm. ... View more

export KAFKA_KERBEROS_PARAMS in /usr/hdp/current/kafka-broker/config/kafka-env.sh you can do this using ambari as well look for kafka-env config template under kafka service. KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/usr/hdp/current/kafka-broker/config/kafka_jaas.conf" ... View more