We need to handle thousands of edge devices running Minifi. We need a solution to hot-upgrade both: replace the current Minifi flow, and/or push a new configuration (e.g. in our case: gps/accelerometer/etc sampling interval) to be used by the currently running Minifi flow.
Our goals are to accomplish these:
1. to update the running Minifi flow. For this we'll try to use C2 Server. Are there better alternatives?
2. to update some custom parameters
a. the Cloud solution should push new sampling interval(s) for gathering edge device's metric(s)
b. Minifi processors to take the new pushed values, and change some of device's run-time settings.
c. [we don't want to restart Nifi processors so we were thinking of using some local/cluster variables. This what we are aiming for.]
Our current (early)research directions:
- create a Minifi process group that receives data, saves them locally, and starts using them
* this is a backup, as it's a simple solution that does not take into account users/groups or ACLs.
- use Zookeeper to share/push configurations from the backend; then use Nifi's ZookeeperStateProvider to link to cluster/group shared data; then use Expression Language's getStateValue to get the updated value.
Could you please help by sharing your thoughts?
Or a hint towards a distributed solution that can handle a hot-configuration-update(s) for Nifi/Minifi processors/variables?