one of the exam objectives of the HDPCD Apache Spark certification is submitting the job to YARN....But to do this we need to build the fat jar, yes I work with Scala...Can someone please clarify what tools during exam we're going to have to build the jar? Are we going to have prepared sbt.build file or is there going to be an empty maven project with dependencies....It's not like with python where you can submit py file....
Thank you very much in advance....exam objective regarding this is very blurry....
During the exam, if you have a question about this, you will be asked to submit a spark application, and in the context of the question, you will be informed the JAR name, the JAR main class and how the application should be processed (local, etc.).
I recently completed certification and it was so.
Hope to help you.
@Gilvan can you please post an example of the code how to run a jar file on yarn. I am confused, in the question will they give me the class name and path to the jar file. my understanding is I have run below script
spark-submit --master yarn --class "whats the class name here" /user/jarfilename.jar
do i also need to type ./bin/spark-submit