Support Questions

Find answers, ask questions, and share your expertise

Tutorial Name Processing Trucking IoT Data with Apache Storm...failed to deploy topology

Explorer

Hi supporters,

i am going through the tutorial in object and while running the deployTopology.sh script i have got the following error:

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre}

2431 [main] INFOo.a.s.StormSubmitter - Finished submitting topology: KafkaToKafka

2431 [main] INFOo.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]

2432 [main] WARNo.a.s.StormSubmitter - Error occurred in invoking submitter hook:[org.apache.atlas.storm.hook.StormAtlasHook]

java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook

at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_141]

at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_141]

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_141]

at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_141]

at java.lang.Class.forName0(Native Method) ~[?:1.8.0_141]

at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_141]

at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]

at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]

at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]

at com.orendainx.hortonworks.trucking.storm.java.topologies.KafkaToKafka.main(KafkaToKafka.java:60) [b7157d00b8f511e78ead0242ac110002.jar:?]

Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook

at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:368)

at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278)

at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390)

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426)

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407)

at com.orendainx.hortonworks.trucking.storm.java.topologies.KafkaToKafka.main(KafkaToKafka.java:60)

Caused by: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:264)

at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361)

... 5 more

Do you have any ideas on how to solve the problem?

Thanks a lot in advance

4 REPLIES 4

@Fabio Ferri,

Did you enable the storm atlas hook?

Go to Storm -> Configs -> Advanced storm-env and Enable Atlas Hook.

39930-screen-shot-2017-10-24-at-110153-pm.png

Thanks,

Aditya

Explorer

Dear Aditya,

thanks for your response.
After enabling the Atlas hook i got i new exception while trying to deploy the topologies:

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre}

2627 [main] INFOo.a.s.StormSubmitter - Finished submitting topology: KafkaToKafka

2627 [main] INFOo.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]

2806 [main] INFOo.a.a.ApplicationProperties - Looking for atlas-application.properties in classpath

2806 [main] INFOo.a.a.ApplicationProperties - Loading atlas-application.properties from file:/etc/storm/2.6.1.0-129/0/atlas-application.properties

log4j:WARN No appenders could be found for logger (org.apache.atlas.ApplicationProperties).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/MapUtils

at org.apache.atlas.security.InMemoryJAASConfiguration.init(InMemoryJAASConfiguration.java:188)

at org.apache.atlas.security.InMemoryJAASConfiguration.init(InMemoryJAASConfiguration.java:177)

at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:65)

at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:69)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:348)

at org.apache.atlas.storm.hook.StormAtlasHook.initialize(StormAtlasHook.java:77)

at org.apache.atlas.storm.hook.StormAtlasHook.<init>(StormAtlasHook.java:46)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.lang.Class.newInstance(Class.java:442)

at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361)

at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278)

at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390)

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426)

at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407)

at com.orendainx.hortonworks.trucking.storm.java.topologies.KafkaToKafka.main(KafkaToKafka.java:60)

Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.MapUtils

at java.lang.ClassLoader.findClass(ClassLoader.java:530)

at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:327)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:135)

... 19 more

Now i have the class "org.apache.commons.collections.MapUtils" which is missing.

Any ideas?

Thanks in advance

@Fabio Ferri,

Can you try doing this

/usr/hdp/current/storm-*/libext/ should contain symlinks for every jar in /usr/hdp/current/atlas-server/hooks/storm/somejavafile.jar

Or you can copy all the jars to /usr/hdp/current/storm-*/libext/ folder and try running it

Thanks,

Aditya

Explorer

Dear Aditya,

i deleted the symlinks and copied the real files into the folder "/usr/hdp/2.6.1.0-129/storm/extlib".

Now i am getting this error while trying to deploy the topologies:

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre}

2370 [main] INFOo.a.s.StormSubmitter - Finished submitting topology: KafkaToKafka

2371 [main] INFOo.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]

Exception in thread "main" java.lang.StackOverflowError

at java.lang.Exception.<init>(Exception.java:102)

at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89)

at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72)

at sun.reflect.GeneratedConstructorAccessor17.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.lang.Class.newInstance(Class.java:442)

at org.apache.atlas.storm.hook.StormAtlasHook.initialize(StormAtlasHook.java:82)

at org.apache.atlas.storm.hook.StormAtlasHook.<init>(StormAtlasHook.java:46)

at sun.reflect.GeneratedConstructorAccessor17.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.lang.Class.newInstance(Class.java:442)

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.