We have tried Metron 0.3.1 in Ubuntu HDP cluster based on the steps provided in the below link,
Its works. But Metron 0.3.x does not have Metron UI so we thought to switch back into Metron 0.4.x.
We have followed the same steps all the topology are started except elasticsearch indexing topology,
Strange class not found exception without much details,
Exception in thread "main" java.lang.RuntimeException: java.lang.ClassNotFoundException: at org.apache.storm.security.auth.AuthUtils.GetAutoCredentials(AuthUtils.java:211) at org.apache.storm.StormSubmitter.populateCredentials(StormSubmitter.java:92) at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:214) at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:310) at org.apache.storm.flux.Flux.runCli(Flux.java:171) at org.apache.storm.flux.Flux.main(Flux.java:98) Caused by: java.lang.ClassNotFoundException: at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.storm.security.auth.AuthUtils.GetAutoCredentials(AuthUtils.java:203)
Please find the attached log for reference.
Metron UI can't able to start in Ubuntu 14.x. Direct me if there are any post related to that.
The "GetAutoCredentials" method where it is failing to find a class, Get all of the configured AutoCredential Plugins so Storm configurations need to be checked.
Based on the error trace it looks like the code is failing to determine the "topology.auto-credentials" based on https://github.com/apache/storm/blob/v1.0.1/storm-core/src/jvm/org/apache/storm/security/auth/AuthUt...
Please check your storm config to findout what is the value set for "topology.auto-credentials" property, Please refer to https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/ref-73...
"topology.auto-credentials" => Set to a list of Java plugins that pack and unpack user credentials for Storm workers. This should be set to org.apache.storm.security.auth.kerberos.AutoTGT. Please check the classes that are set to this property are included in the classpath.
Thanks @Jay Kumar SenSharma
I have added "topology.auto-credentials"property in ambari but no luck,
Do we first configure storm for that "Configuring Authorization for Storm"
Is the ambari will take of this " Please check the classes that are set to this property are included in the classpath." ?
Please guide me on this.
Thanks @Jay Kumar SenSharma
I have fixed the issue and elasticsearch indexing topology now working. The issue because of in elasticsearch.properties file,
It is provided empty if we are enabled kerberos then this has to be set with AutoTGT. If not just removing that property value will fix the issue,
In all the other properties value for topology.auto-credentials is empty other than elasticsearch.properties..
By default it should be provided as empty like in the other properties.