Created 06-30-2016 09:24 PM
how to setup oozie custom share lib for an action in places other then sharelib_timestamp and reference them with a name so then later we can use this in the configuration ? We are trying to stage a shared lib directory outside of the normal /users/oozie/share/libs/XXXXXXXXX
Assumption is that oozie.service.ShareLibService.mapping.file is what will acccomplish this for us
e.g. I already have
But I want to create another directory for specific spark version and call use this as share lib for my spark action.
Created 06-30-2016 11:54 PM
You can put your custom jars in /user/oozie/share/lib/spark_161 and add below line in your job.properties
to reference them
Created 07-01-2016 12:08 AM
Hi @Kuldeep Kulkarni can this "oozie.libpath" be used in conjunction to oozie.use.system.libpath=true. Also, how can infrastructure team enforce this for a particular action without tenants needing to change their job.properties. How does "oozie.service.ShareLibService.mapping.file" work? can this be used instead?
Created 07-01-2016 05:36 PM
1. Yes you can use both in job.properties.
2. If you want it to be workflow specific then there is an alternative way
- No need to add anything in job.properties or sharelib. You can just create a directory named “lib” next to your workflow.xml in HDFS and put jars in there.
3. Regarding oozie.service.ShareLibService.mapping.file --> This needs to be configured in oozie-site.xml. need service restart, I would suggest go with option 1 OR 2
Hope this information helps.
Please accept the answer if you got required information :)
Created 07-01-2016 08:03 PM
@Kuldeep Kulkarni I understand these options. maybe I have not added my requirement explicitly. Oozie server uses following sharlib by default for respective Action Name. /user/oozie/share/lib/lib_$epochtime/$ActionName
where $epochtime -> latest timestamp upon reboot. How do we change this default location from Oozie server side to be more deterministic manner, ($epochtime is variable). Ok to restart Oozie server when there is a change, seamless to the user submitting the job. (dont want user to update their job.properties or workflow directories)
Created 07-08-2016 03:07 PM