- last edited on
We are trying to run a spark program using NiFi. [This is the basic sample] we tried to follow.
We have configured Apache-Livy server in
ExecutiveSparkInteractive processor is used to run sample Spark code.
val gdpDF = spark.read.json("gdp.json")
val gdpRDD = gdpDF.rdd
LivyController is configured for 127.0.0.1 port 3998 and Session Type : spark.
When we run the processor we get following error :
Spark Session returned an error, sending the output JSON object as the flow file content to failure (after penalizing)
We just want to output the line count in JSON file. How to redirect it to flowfile?