Run Kafka+Storm in a Kerberized HDP cluster

[#1]

In a Kerberized HDP cluster, I use FreeIPA as LDAP and run HDP 2.3.2 including Storm, Kafka and Ranger. After creating a storm topology as a normal user, I keep getting the following runtime error:

2015-12-30 18:31:53.274 o.a.c.ConnectionState [ERROR] Authentication failed 
... 
2015-12-30 18:31:53.286 o.a.z.ClientCnxn [WARN] SASL configuration failed: javax.security.auth.login.LoginException: No password provided Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 
... 
2015-12-30 18:31:53.328 b.s.util [ERROR] Async loop died! java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode ... 

This problem comes from Kafka and Ranger.

As a normal failure recovery mechanism, Kafka will keep creating the topic if it cannot find the designated topic mentioned in the Kafka producer. In a kerberized environment, this CREATE request will be sent to Ranger for approval. However, in HDP 2.3.2, Ranger 0.5.0.2.3 cannot understand/recognize the CREATE action from Kafka. Therefore, this request is blocked.

This problem in Ranger is fixed in the latest HDP 2.3.4. In order to solve this problem temporarily, you only need to restart Kafka. It also helps to have a more powerful cluster and larger memory. This solution is still valid in HDP 2.5.

Thanks to Sumit Mohanty, Madhan Neethiraj and Sriharsha Chintalapani for the kind help!