Can NiFi help other technologies perform better?

[#1]

Hi, i am new to HDF but am very interested in NiFi. There are a few questions i would like to ask about NiFi.

1; How does NiFi get the data from IoT devices? My understanding is that they send the data to a folder on the machine running NiFi and it uses a getFile processor to retrieve it. Is this incorrect.

2: Can you perform enough processing on NiFi that would ease the workload on other technologies such as spark etc, enabling them to perform tasks quicker?

3: Where does MiniFi fit into the NFi scenario?

I have read many documents and watched videos but still need a litlle help in understanding certain aspects of NiFi. I would appreciate any input on these questions, thank you.

[#2 · Accepted solution]

1) In many cases, the edge device would run MiNiFi and push the data to a NiFi processing cluster using the site-to-site protocol

2) Perhaps, but the core issue is arriving at the right design pattern and the separation of concerns between dataflow management and streaming analytics. Many streaming use cases require functionality that is rooted in mediation and exchange of data, as well as complex event processing and computation on streams. NiFi is the natural home for the former concerns.

3) See 1)

[#3]

Ok thank you, i was guessing thats what MiNiFi did. Thanks for your answers, very helpful

[#4]

1. Currently, MiNiFi C++ can use TailFile/GetFile or ListenSyslog processors to ingest data. In the future, more IoT-focused processors will be made available to communicate directly using the protocol specified by the edge device. At this time, MiNiFi Java has access to more processors because of the common development platform.

2. @slachterman's answer is very good.

3. I describe some of the interaction and use cases for NiFi and MiNiFi here.

[#5]

Hi @Andy LoPresto, thank you for the info. i have just read the link on Stack overflow. it is an excellent description and very informative.

[#6]

Glad it helped, Roger. I just updated the answer with a link to the list of current MiNiFi Java processors.