What is the Storm kerberos support version?

[#1]

Hello~ I am working in kerberos cluster..

My Environment is

Ambari : 2.4.2.0

HDP 2.5.3.0

- Storm : 1.0.1

- Kafka : 0.10.0

Of course I finished kerberos cluster by using ambari

All daemons( for example storm, hdfs, hbase , kafka and so on) are ok. And I don't see any error log in daemon logs

After finishing the kerberos cluster.. I deployed Storm jar to the kerberos cluster

The storm start command is not working..

So I searched many things why the error was happended..

Lastly I found the article in the website

"The storm version which support kerberos is 1.1.0"

Is is right? and the storm version 1.0.1 is not supporting kerberos?

Thank you in advanced..

[#2]

@sk rho Is the storm service not coming up? Or is it topology that is having problems running?

If it is the topology that is having problem you are refer the below answer:

To allow user to deploy a storm topology in secure mode

1. create user level storm.yaml under /home/username/.storm/storm.yaml . This storm.yaml can only contain few required settings or you can just copy the entire storm.yaml from /etc/storm/conf/storm.yaml

required settings if you are adding are

1. nimbus.seeds

2. storm.thrift.transport: "org.apache.storm.security.auth.kerberos.KerberosSaslTransportPlugin"

3. java.security.auth.login.config: "/etc/storm/conf/client_jaas.conf"

4. Make sure /etc/storm/conf/client_jaas.conf contents looks like this

StormClient{

com.sun.security.auth.module.Krb5LoginModule required

doNotPrompt=false

useTicketCache=true

serviceName="nimbus";

};

5. Run kinit with user's keytab before running storm jar command

By default Storm uses SimpleACLAuthorizer as authorizer to give permissions to user. This will allow all user's with valid keytab to submit topologies.

- If the service is not starting up can you please send the error message that you see in ambari and also the logs for storm.

[#3]

Thank your answer~

But The Error is stall alive~ ㅠ.ㅠ

Please let me know about the right answer

Error Message is

2018-02-01 13:43:42.894 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=topic_inbound_master, partitionMap={0=null:-1, 1=null:-1, 2=null:-1, 3=null:-1, 4=null:-1, 5=null:-1, 6=null:-1, 7=null:-1, 8=null:-1, 9=null:-1, 10=null:-1, 11=null:-1, 12=null:-1, 13=null:-1, 14=null:-1, 15=null:-1, 16=null:-1, 17=null:-1, 18=null:-1, 19=null:-1}} 2018-02-01 13:43:42.894 o.a.s.k.KafkaUtils [INFO] Task [7/10] assigned [Partition{host=null:-1, topic=topic_inbound_master, partition=6}, Partition{host=null:-1, topic=topic_inbound_master, partition=16}] 2018-02-01 13:43:42.894 o.a.s.k.ZkCoordinator [INFO] Task [7/10] Deleted partition managers: [] 2018-02-01 13:43:42.895 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=topic_inbound_master, partitionMap={0=null:-1, 1=null:-1, 2=null:-1, 3=null:-1, 4=null:-1, 5=null:-1, 6=null:-1, 7=null:-1, 8=null:-1, 9=null:-1, 10=null:-1, 11=null:-1, 12=null:-1, 13=null:-1, 14=null:-1, 15=null:-1, 16=null:-1, 17=null:-1, 18=null:-1, 19=null:-1}} 2018-02-01 13:43:42.895 o.a.s.k.KafkaUtils [INFO] Task [1/10] assigned [Partition{host=null:-1, topic=topic_inbound_master, partition=0}, Partition{host=null:-1, topic=topic_inbound_master, partition=10}] 2018-02-01 13:43:42.895 o.a.s.k.ZkCoordinator [INFO] Task [1/10] Deleted partition managers: [] 2018-02-01 13:43:42.895 o.a.s.k.ZkCoordinator [INFO] Task [1/10] New partition managers: [Partition{host=null:-1, topic=topic_inbound_master, partition=0}, Partition{host=null:-1, topic=topic_inbound_master, partition=10}] 2018-02-01 13:43:43.829 o.a.h.u.NativeCodeLoader [WARN] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-02-01 13:43:43.964 o.a.s.k.PartitionManager [INFO] Read partition information from: /brokers/topics/topic_inbound_master/kafkaspout1/partition_2 --> null 2018-02-01 13:43:43.967 o.a.s.k.PartitionManager [INFO] Read partition information from: /brokers/topics/topic_inbound_master/kafkaspout1/partition_4 --> null 2018-02-01 13:43:43.964 o.a.s.k.PartitionManager [INFO] Read partition information from: /brokers/topics/topic_inbound_master/kafkaspout1/partition_6 --> null 2018-02-01 13:43:43.972 o.a.s.k.PartitionManager [INFO] Read partition information from: /brokers/topics/topic_inbound_master/kafkaspout1/partition_8 --> null 2018-02-01 13:43:43.975 o.a.s.k.PartitionManager [INFO] Read partition information from: /brokers/topics/topic_inbound_master/kafkaspout1/partition_0 --> null 2018-02-01 13:43:44.131 k.c.SimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2018-02-01 13:43:44.131 k.c.SimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2018-02-01 13:43:44.131 k.c.SimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2018-02-01 13:43:44.133 k.c.SimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2018-02-01 13:43:44.134 k.c.SimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2018-02-01 13:43:44.134 o.a.s.util [ERROR] Async loop died! java.lang.RuntimeException: java.nio.channels.ClosedChannelException at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:103) ~[stormjar.jar:?] at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[stormjar.jar:?] at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:129) ~[stormjar.jar:?] at org.apache.storm.daemon.executor$fn__6505$fn__6520$fn__6551.invoke(executor.clj:651) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37] at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77] Caused by: java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) ~[stormjar.jar:?] at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:78) ~[stormjar.jar:?] at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$sendRequest(SimpleConsumer.scala:68) ~[stormjar.jar:?] at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:127) ~[stormjar.jar:?] at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79) ~[stormjar.jar:?] at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:75) ~[stormjar.jar:?] at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:65) ~[stormjar.jar:?] at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:94) ~[stormjar.jar:?] at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) ~[stormjar.jar:?] ... 6 more