Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 07-07-2016 03:28 PM
Hi, I am beginner in Storm, For submitting storm topology i am using these below set of lines
conf.put(Config.NIMBUS_HOST, NIMBUS_NODE);
conf.put(Config.NIMBUS_THRIFT_PORT,6627);
StormSubmitter.submitTopology("test", conf, builder.createTopology()); and in my pom, we have storm-core-0.9.3 and i am using HDP2.2.
Now we are migrating to HDP2.4 which is using storm-core-0.10.0 so i changed my pom to storm-core-0.10.0 but i am getting error as Required field 'nimbus_uptime_secs' is unset!
In code i changed/added
conf.put(Config.NIMBUS_HOST, NIMBUS_NODE); conf.put(Config.NIMBUS_THRIFT_PORT,6627);
ClusterSummary cs = new ClusterSummary();
cs.setFieldValue(_Fields.NIMBUS_UPTIME_SECS, 2);
StormSubmitter.submitTopology("test", conf, builder.createTopology());
but not able to figure out how to pass the setted value to StormSubmitter. I tried moving to storm-core-1.0.1 and storm-core-0.11.0-snapshot but those are causing me compilation issues in other dependant code. Thank you
Created 07-07-2016 04:13 PM
Storm in HDP 2.4 is based on Apache Storm 0.10.0 but not same since it has Nimbus H/A feature which is available on Apache Storm 1.0.0.
You can change your storm-core dependency to match your Storm version like 0.10.0-2.4.x.xxx according to your HDP version and see it helps.
You may also want to add Hortonworks repository (http://repo.hortonworks.com/content/repositories/releases) to your maven pom or gradle config.
Created 07-07-2016 04:13 PM
Storm in HDP 2.4 is based on Apache Storm 0.10.0 but not same since it has Nimbus H/A feature which is available on Apache Storm 1.0.0.
You can change your storm-core dependency to match your Storm version like 0.10.0-2.4.x.xxx according to your HDP version and see it helps.
You may also want to add Hortonworks repository (http://repo.hortonworks.com/content/repositories/releases) to your maven pom or gradle config.
Created 07-09-2016 10:48 AM
Thank you Lim, I already tried what you have mentioned but my scenerio is we are using the client which is apache storm ie, /Users/samrat/dev/StormCluster/apache-storm-0.10.0/bin/storm jar sample.jar com.bigd.tool.SampleMain ..... but now I think i should change my code to be compatible with apache-storm 1.0.1. Thank you once again Lim for your response
Created 07-14-2016 02:22 PM
I zipped the storm from sandboxHDP2.4 and unzipped it in my local machine and created two folders as /usr/hdp/2.4.0.0-169/etc and /usr/hdp/2.4.0.0-169/storm. Created default folder in etc and copied hadoop file and created bin folder in storm and copied storm.distro and storm.py. I keep on getting some or other file missing like
No such file or directory: '/usr/hdp/2.4.0.0-169/storm/extlib' Could somebody help me is there any better solution or other procedure. I can't go to apache storm 1.0.1 as hdp2.4 has storm-0.10.0-2.4.0.0-169. Basically I want to submit job from storm client which is present in my local machine to cluster which is in sandbox