Support Questions

Find answers, ask questions, and share your expertise

how to manage dependent jars for oozie job

avatar
Expert Contributor

Hi, everyone

 

when we deploy spark or java , sqoop job in oozie by Hue, mostly we need add jar files to resolve the denpendent jars issue.

 

but sometimes some jobs need many jars , we need to run and run job again to discover the missed jar from oozie log, and add these jars file.

 

i am thinking many times how to become more effient and convenicent. who have a good idea for this ?

 

 

2 REPLIES 2

avatar
Super Guru

avatar
Expert Contributor

thanks.

 

my pain point is when i finished a spark , java, hive, sqoop job, i tested it on Linux System , mostly it's normal.

 

then i deploy the job in oozie by Hue, it always throw some errors like Class not found . i know it because some jars missed, then i am going to search which jars is missed, if found , i add this jar file.

 

so i mean i don't know which jar files is needed while i deploy these job in oozie, i need the try and try to found missed jars.  this is my pain point.

 

these jobs executed on Linux System model is normal  i think it's because the job load all the classpath jar files, so it's no this kind issue.