Spark Structured Streaming vs NIFI

[#1]

Hi

Spark 2.0 now had structured streaming. How it is different from NIFI file streaming.

[#2 · Accepted solution]

@Satya KONDAPALLI

Fundamentally, Spark is a data processing engine while NiFi is a data movement tool. Spark is intended for doing complex computations on large amounts of data, combining data sets, applying analytical models, etc. Spark Streaming provides micro batch processing of data to bring this processing closer to real time. NiFi is intended to collect data and move it to the place for it to be processed with some certain modifications or computations on the data as it flows to its final destination.