Created 06-01-2016 10:40 AM
I want to pass some custom configuration properties to storm topology that are not part of storm yaml, how can I pass it.
Created 06-01-2016 10:50 AM
you can use following ways to get the external configuration inside the topplogy
1: pass the arguments like this storm jar storm-jar topology-name -c sKey=sValue -c key1=value1 -c key2=value2 >/tmp/storm.txt 2: Create a simple java resource file (properties files) and pass it as arguments to your topology main class, in main method read the properties from the main file and build the storm configuration object using conf.put() 3: create separate yaml file read it through the Utils method provided by storm api,look for more documentation https://nathanmarz.github.io/storm/doc/backtype/storm/utils/Utils.html Utils.findAndReadConfigFile() |
Created 06-01-2016 10:50 AM
you can use following ways to get the external configuration inside the topplogy
1: pass the arguments like this storm jar storm-jar topology-name -c sKey=sValue -c key1=value1 -c key2=value2 >/tmp/storm.txt 2: Create a simple java resource file (properties files) and pass it as arguments to your topology main class, in main method read the properties from the main file and build the storm configuration object using conf.put() 3: create separate yaml file read it through the Utils method provided by storm api,look for more documentation https://nathanmarz.github.io/storm/doc/backtype/storm/utils/Utils.html Utils.findAndReadConfigFile() |
Created 06-01-2016 01:43 PM
Consider wiring your topologies with Storm's Flux. It also allows for property substitution: http://storm.apache.org/releases/current/flux.html
Created 11-03-2016 11:55 AM
You can use
Config conf = new Config();
conf.put(key, val);
conf.put(key1, val1);
e.g redis config, etc.
and then you can use this in prepare method of bolts(in case of trident in functions/ filters, etc).
Hope this helps.
Created 06-07-2017 09:08 PM
@Andrew Grande, @Rajkumar Singh
I am trying to make the -c work with Flux and the exec-maven-plugin:
call mvn compile exec:java -Dexec.args="--local --sleep 400000 --resource /topology.yaml -c foo.bar=1"
, but I get:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project StorageTopology: An exception occured while executing the Java class. null: InvocationTargetException: Unrecognized option: -c -> [Help 1]
Does this only work with storm jar submission?
Can I make it work with the exec maven plugin?
Thx,
Mauro.