what is the relation between nifi and storm in HDF?

[#1]

It seems that storm is mentioned rarely in HDF.

Can nifi replace storm?

Nifi and storm is independent to each other?(no processer in nifi communicate with storm )

[#2]

I suggest having a gander at related comments here.

NiFi cannot replace Storm for some use cases. NiFi can replace Storm for some use cases.

NiFi's sweet spot is simple event processing (one event context at a time). Storm, Spark, Flink, etc. are all more powerful when it comes to complex (windowed, or cross-event) processing, but require writing code.

Do you have a particular use case in mind?

[#3 · Accepted solution]

HDF, as a data-in-motion platform, contains NIFI as one component, for data ingestion & flow management, and Storm as another component for complex event processing.

NIFI and Storm are complementary to each other. NIFI is not yet another processing framework, but to deliver data to those frameworks such as Storm. Both NIFI and Storm can handle processing jobs, but we tend to leverage NiFi for Simple Event Processing, whereas Storm for Complex Event Processing. In terms of Simple Event Processing, usually you are able to operate on a single piece of data by itself (or joining with an Enrichment Dataset). Complex Event Processing typically normally refers to processing data from multiple streams, think of a JOIN operation, or rolling window aggregation type of analytics.

You can embed NIFI-external-modules into Storm for integration purposes, in order to leverage both components to achieve your ultimate streaming analytics goals. As an example, you can either embed a "NiFiSpout" into Storm to pull data from NIFI, or "NiFiBolt" to send data back to NIFI

[#4]

thank you!