Interesting... In regards to flow design do you mean if a node go's down replay everything that has not been acknowledged as processed at your data source (jms, kafka) and then remove duplicates before pushing to the data's storage destination (kafka, HDFS). I think the Notify, Wait and DetectDuplicate processors would be usfeul in this case. ... View more

Also.... Is there estimation for when High Availability Processing will be available or is there a work around that could be put in place? ... View more

The data is time critical and must be as near to real time as possible. If a node goes down and there is a delay in that nodes queued data getting to its destination this would not be acceptable as alerts could potentially be delayed. Could you please describe the process for migrating queued data of a failed node to a new node? My thoughts are this could potentially be automated and might fit into an acceptable time frame for data to get to its destination. ... View more