question nifi - spark streaming in Archives of Support Questions (Read Only)

nifi - spark streaming

avijeetd — Thu, 08 Dec 2016 15:38:43 GMT

Hi,

I saw some notes on that we can use site-to-site to trigger spark streaming, also saw a note on nifi-spark-receiver jar

can someone pls suggest whats the best way to connect nifi and spark streaming.

Thanks,

Avijeet

Re: nifi - spark streaming

andrewg — Thu, 08 Dec 2016 21:02:43 GMT

The recommended setup for production is to use Kafka. NiFi publishes to Kafka, Spark Streaming consumes from the topic (or the reverse). Spark Receiver in NiFi works, but wasn't tested at production scale.

Re: nifi - spark streaming

sunnybhan1501 — Thu, 02 Mar 2017 21:18:46 GMT

I am using Spark receiver for Nifi.. its not working..

# Site to Site properties nifi.remote.input.host=localhost nifi.remote.input.secure=false nifi.remote.input.socket.port=10000 nifi.remote.input.http.enabled=true nifi.remote.input.http.transaction.ttl=30 sec