when we deploy spark or java , sqoop job in oozie by Hue, mostly we need add jar files to resolve the denpendent jars issue.
but sometimes some jobs need many jars , we need to run and run job again to discover the missed jar from oozie log, and add these jars file.
i am thinking many times how to become more effient and convenicent. who have a good idea for this ?
my pain point is when i finished a spark , java, hive, sqoop job, i tested it on Linux System , mostly it's normal.
then i deploy the job in oozie by Hue, it always throw some errors like Class not found . i know it because some jars missed, then i am going to search which jars is missed, if found , i add this jar file.
so i mean i don't know which jar files is needed while i deploy these job in oozie, i need the try and try to found missed jars. this is my pain point.
these jobs executed on Linux System model is normal i think it's because the job load all the classpath jar files, so it's no this kind issue.