question Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset! in Archives of Support Questions (Read Only)

Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

halshater — Wed, 06 Jan 2016 18:16:29 GMT

Hi, I'm trying to submit a topology on storm cluster using StormSubmitter , however I got the following error: [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project realtime_piplines: An exception occured while executing the Java class. null: InvocationTargetException: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! I am running HDP 2.3.4 with Storm version '0.10.0' on RHEL 6.5.

Thanks in advance,

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

aervits — Wed, 06 Jan 2016 21:52:46 GMT

@Hani Al-Shater can you paste the command you use to execute the topology and paste the supervisor logs?

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

ychemolosov — Fri, 08 Jan 2016 03:43:21 GMT

I am experiencing the same issue with freshly installed HDP 2.3 cluster.

I guess, it has been fixed in Storm re: https://issues.apache.org/jira/browse/STORM-1189 but only available in ver. 0.11.0

What would be the best way get my topology submitted to storm ? I there a nice way to patch or downgrade/upgrade the system?

I use steamparse to submit the topology to the Storm.

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

aervits — Fri, 08 Jan 2016 10:49:24 GMT

@Yuri Chemolosov @schintalapani

good find Yuri, have you tried the suggested work around and set nimbus_uptime_secs to 0?

the https://storm.apache.org/apidocs/backtype/storm/generated/ClusterSummary.html#set_nimbus_uptime_secs(int)

let me know if the workaround works, I see it is happening frequently.

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

schintalapani — Fri, 08 Jan 2016 11:34:23 GMT

@Yuri Chemolosov @Artem Ervits one of the workarounds I think of is to make storm-0.11-SNAPSHOT as dependency at the client side for the topology and use the StormSubmitter for 0.11-SNAPSHOT version.

add the following repository in your pom.xml https://repository.apache.org/content/repositories/snapshots/ to get the latest snapshot for storm.

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

aervits — Fri, 08 Jan 2016 11:50:53 GMT

@Hani Al-Shater try this and let us know

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

ychemolosov — Sat, 09 Jan 2016 03:19:59 GMT

I have used 0.11.0-SNAPSHOT storm-core to submit the task as suggested. It acutaly passes the submission phase (no more original error). The topology get deployed to the cluster. But then it fails to start with some incompatibility exception on the server side.

Is there a way to selectively upgrade servers side Storm to the snapshot version?

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

aervits — Sat, 09 Jan 2016 03:38:47 GMT

@Yuri Chemolosov there's no way to upgrade individual components aside from Spark.

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

ychemolosov — Sat, 09 Jan 2016 12:32:52 GMT

Tried everything exhausting the options. Nothing worked, neither new client, nor old. Locally rebuilt patched storm-core did not work either.

Had to reinstall the whole cluster thing downgrading to HDP 2.2, moving to Ubuntu 12 from Debian 7, resolving all the failed dependency of the OS.

Now I am still getting the 'incompatibility error'

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

aervits — Sun, 10 Jan 2016 09:47:30 GMT

@Yuri Chemolosov can you confirm storm-starter can work on your cluster compiled with the version of your HDP cluster?

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

halshater — Sun, 10 Jan 2016 18:07:23 GMT

Thanks all,

Changing storm version (by downgrading), setting the nimbus_uptime_secs did not work for me. However, I have managed to deploy the topology remotly, 2 things I did:

1. Used storm client instead of run the topology using mvn exec:java

2. Removed configuration from source code and configure ~/.storm/storm.yaml instead.

Now I have hive, hdfs bolts and kafka spout only working in the local mode, I will investigate, help is appreciated.

Thanks again

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

ychemolosov — Tue, 12 Jan 2016 23:15:35 GMT

Solution is to tell your build/submit tool to use the storm library from HDP repository. Here how it is described for streamparse

Re: Submitting a topology to storm cluster - Required field 'nimbus_uptime_secs' is unset!

halshater — Thu, 14 Jan 2016 19:53:49 GMT

Thank you, it worked fine.