Hi , I am using cloudera CDP Public cloud with CDP Runtime version 7.2.17. As Apache storm is not a part of cloudera stack, i downloaded from apache website and configured kerberos manually with the help of storm documentation https://storm.apache.org/releases/2.6.1/SECURITY.html I am using kafka spout and hdfs bolt , I was able to connect to kafka spout with kafka Spout configuration properties, But i faced SIMPLE authentication is not enabled. Available:[TOKEN] error for hdfs bolt by going through storm documentation related to hdfs bolt( https://storm.apache.org/releases/2.1.0/storm-hdfs.html ) i configured some of properties in yaml file storm.yaml storm.zookeeper.servers:
- "hari-cluster-test1-master0.avinash.ceje-5ray.a5.cloudera.site"
storm.local.dir: "/home/venkataavinash_kesineni/storm/apache-storm-2.6.1/data"
nimbus.seeds: ["hari-cluster-test1-master0.avinash.ceje-5ray.a5.cloudera.site"]
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
- 6704
- 6705
storm.thrift.transport: "org.apache.storm.security.auth.kerberos.KerberosSaslTransportPlugin"
nimbus.childpots: "-Xmx1024m -Djava.security.auth.login.config=/home/venkataavinash_kesineni/storm/apache-storm-2.6.1/conf/jaas.conf"
ui.childpots: "-Xmx1024m -Djava.security.auth.login.config=/home/venkataavinash_kesineni/storm/apache-storm-2.6.1/conf/jaas.conf"
supervisor.childopts: "-Xmx256m -Djava.security.auth.login.config=/home/venkataavinash_kesineni/storm/apache-storm-2.6.1/conf/jaas.conf"
storm.zookeeper.superACL: "sasl:storm"
java.security.auth.login.config: "/home/venkataavinash_kesineni/storm/apache-storm-2.6.1/conf/jaas.conf"
nimbus.autocredential.plugins.classes: ["org.apache.storm.hdfs.security.AutoHDFSNimbus"]
nimbus.credential.renewers.classes: ["org.apache.storm.hdfs.security.AutoHDFSNimbus"]
hdfs.keytab.file: "/root/hdfs.keytab"
hdfs.kerberos.principal: "hdfs/hari-cluster-test1-master0.avinash.ceje-5ray.a5.cloudera.site@AVINASH.CEJE-5RAY.A5.CLOUDERA.SITE"
nimbus.credential.renewers.freq.secs: 82800
topology.auto-credentials: ["org.apache.storm.hdfs.security.AutoHDFS"] When i am submitting topology i am getting the following error Error java.lang.RuntimeException: org.apache.storm.thrift.TApplicationException: Internal error processing submitTopologyWithOpts
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:304) ~[storm-client-2.6.1.jar:2.6.1]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:212) ~[storm-client-2.6.1.jar:2.6.1]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:175) ~[storm-client-2.6.1.jar:2.6.1]
at com.voltage.securedata.stream.storm.StormTopology.main(StormTopology.java:220) [vs-kafka-storm-1.0.jar:?]
Caused by: org.apache.storm.thrift.TApplicationException: Internal error processing submitTopologyWithOpts
at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:81) ~[storm-shaded-deps-2.6.1.jar:2.6.1]
at org.apache.storm.generated.Nimbus$Client.recv_submitTopologyWithOpts(Nimbus.java:376) ~[storm-client-2.6.1.jar:2.6.1]
at org.apache.storm.generated.Nimbus$Client.submitTopologyWithOpts(Nimbus.java:359) ~[storm-client-2.6.1.jar:2.6.1]
at org.apache.storm.StormSubmitter.submitTopologyInDistributeMode(StormSubmitter.java:354) ~[storm-client-2.6.1.jar:2.6.1]
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:292) ~[storm-client-2.6.1.jar:2.6.1]
... 3 more
Please let me know if I need to do any other steps/config related changes for storm-hdfs bolt to work with Kerberos.
... View more
