Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi - Execute spark Scala Jar

Highlighted

Nifi - Execute spark Scala Jar

New Contributor

Hello, i need execute a spark job from NiFi. The file .jar is in HDFS and i need know if the job finish success or fail.

What is the best option?

it's possible launch spark-submit command and receive the status?

Thanks

2 REPLIES 2

Re: Nifi - Execute spark Scala Jar

Rising Star

Yes you can run ephemeral spark jobs via NiFi please refer https://community.hortonworks.com/repos/64179/launching-spark-jobs-from-nifi.html and https://github.com/diegobaez/PUBLIC/tree/master/NiFi-SnapSpark

Note: Please upvote or mark this answer accepted if it resolves you found it useful

Re: Nifi - Execute spark Scala Jar

New Contributor

@PankajKadam The spark that i need launch is indepent of the flow, I mean, I need to launch a spark-submmit command and receive the status without flowfile.

Thanks