Can we configure spark processor in NIFI?
Hi @hema m,
You can use the "execute process" processor to start a spark job.
if you use execute process make sure it's YARN mode so that it doesn't hang and wait.
Run it in YARN Cluster mode from the client spark-submit.
I have used:
Site-to-Site Protocol (S2S)
NiFi puts to Kafka, Spark Streaming ingest Kafka messages