Skip to main content
jarededrake
jarededrake
Silver ⭐️⭐️⭐️
April 12, 2022
Question

Oozie work flow not finding main class

Hello Community,

 

I am going through this tutorial https://www.cloudera.com/tutorials/setting-up-a-spark-development-environment-with-java.html and I am trying to submit it through Oozie work flow. In Hue I am going to Query > Scheduler > Workflow then dropping Java program to actions and then uploading jar file then adding the main class which is Hortonworks.SparkTutorial.Main. When I click to run the Oozie work flow job I keep getting the error of: "Caused by: java.lang.ClassNotFoundException: Class Hortonwork.SparkTutorial.Main not found". I'm using intelliJ to do this project so I hold down ctrl and hover over Main and it takes me to MANIFEST.MF file and it says Main-Class: Hortonworks.SparkTutorial.Main so I'm getting the main class definition right I feel like. I cannot figure out why it is saying it can't find my class.

 

 

10 replies

mszurap
Platinum ⭐️
mszurap
Platinum ⭐️
April 13, 2022

Hi @jarededrake , The "ClassNotFoundException: Class Hortonwork.SparkTutorial.Main not found" suggests that in the Java program's main class package name might have a typo (in your workflow definiton), the Hortonwork should be Hortonworks. Can you check that?

Translate
jarededrake
jarededrakeAuthor
Silver ⭐️⭐️⭐️
April 14, 2022

@mszurap You were right I didn't have the "s" at the end 😐. However I am still getting the problem on oozie, I've add a screen shot so you can see the structure of my program and you can see in the MANIFEST.MF file it has my Main class as 

Hortonworks.SparkTutorial.Main


Capture9.PNG
Translate
mszurap
Platinum ⭐️
mszurap
Platinum ⭐️
April 14, 2022

I see. Have you verified that the built jar contains this package structure and class names? Can you also show where the jar is uploaded and how is it referenced in the oozie workflow?

Thanks, Miklos

Translate
jarededrake
jarededrakeAuthor
Silver ⭐️⭐️⭐️
April 26, 2022

Hey @mszurap , no problem at all totally understand.  So I ran my "SparkTutorial-1.0-SNAPSHOT.jar" from Hue > Query > Scheduler > Workflow, I then drag down Spark to the work flow, I then fill out the inputs like this: 

Jar/py name: SParkTutorial-10.0.SNAPSHOT.jar

Main class: Main 

Files + : /user/zzmdrakej2/SparkTutorial-1.0-SNAPSHOT.jar

 

I then get this error, its different so I guess that's a good sign right: 

 

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, Delegation Token can be issued only with kerberos or web authentication

 

 

Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
May 3, 2022

@jarededrake Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!

Translate
Regards, Diana Torres, Senior Community Moderator
Terms & ConditionsAccessibility statement