Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to manage dependent jars for oozie job

avatar
Expert Contributor

Hi, everyone

 

when we deploy spark or java , sqoop job in oozie by Hue, mostly we need add jar files to resolve the denpendent jars issue.

 

but sometimes some jobs need many jars , we need to run and run job again to discover the missed jar from oozie log, and add these jars file.

 

i am thinking many times how to become more effient and convenicent. who have a good idea for this ?

 

 

2 REPLIES 2

avatar
Super Guru

avatar
Expert Contributor

thanks.

 

my pain point is when i finished a spark , java, hive, sqoop job, i tested it on Linux System , mostly it's normal.

 

then i deploy the job in oozie by Hue, it always throw some errors like Class not found . i know it because some jars missed, then i am going to search which jars is missed, if found , i add this jar file.

 

so i mean i don't know which jar files is needed while i deploy these job in oozie, i need the try and try to found missed jars.  this is my pain point.

 

these jobs executed on Linux System model is normal  i think it's because the job load all the classpath jar files, so it's no this kind issue.