I was wondering what else could be the cause of this error. Jar not found then Class not found exception. I verified that the properties file are pointing to the correct locations and also the lib jar file is in the proper location in hdfs like I literally checked where the oozie workflow claimed it to be and it was in the right spot.
As well I ran the spark submit from the command line outside of the oozie workflow and it was successful.
This is the first time I encountered this error I had used this format already many times before and everything worked.
I just wanted to know if there is any other problem that could be causing this error.
I am fine with even general answers. I looked elsewhere but was not able to find a good answer to this question.