Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark Structured Streaming vs NIFI

Solved Go to solution
Highlighted

Spark Structured Streaming vs NIFI

New Contributor

Hi

Spark 2.0 now had structured streaming. How it is different from NIFI file streaming.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Spark Structured Streaming vs NIFI

@Satya KONDAPALLI

Fundamentally, Spark is a data processing engine while NiFi is a data movement tool. Spark is intended for doing complex computations on large amounts of data, combining data sets, applying analytical models, etc. Spark Streaming provides micro batch processing of data to bring this processing closer to real time. NiFi is intended to collect data and move it to the place for it to be processed with some certain modifications or computations on the data as it flows to its final destination.

View solution in original post

1 REPLY 1
Highlighted

Re: Spark Structured Streaming vs NIFI

@Satya KONDAPALLI

Fundamentally, Spark is a data processing engine while NiFi is a data movement tool. Spark is intended for doing complex computations on large amounts of data, combining data sets, applying analytical models, etc. Spark Streaming provides micro batch processing of data to bring this processing closer to real time. NiFi is intended to collect data and move it to the place for it to be processed with some certain modifications or computations on the data as it flows to its final destination.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here