Created 06-30-2016 09:24 PM
how to setup oozie custom share lib for an action in places other then sharelib_timestamp and reference them with a name so then later we can use this in the configuration ? We are trying to stage a shared lib directory outside of the normal /users/oozie/share/libs/XXXXXXXXX
Assumption is that oozie.service.ShareLibService.mapping.file is what will acccomplish this for us
e.g. I already have
But I want to create another directory for specific spark version and call use this as share lib for my spark action.
You can put your custom jars in /user/oozie/share/lib/spark_161 and add below line in your job.properties
to reference them
Created 07-01-2016 12:08 AM
Hi @Kuldeep Kulkarni can this "oozie.libpath" be used in conjunction to oozie.use.system.libpath=true. Also, how can infrastructure team enforce this for a particular action without tenants needing to change their job.properties. How does "oozie.service.ShareLibService.mapping.file" work? can this be used instead?
1. Yes you can use both in job.properties.
2. If you want it to be workflow specific then there is an alternative way
- No need to add anything in job.properties or sharelib. You can just create a directory named “lib” next to your workflow.xml in HDFS and put jars in there.
3. Regarding oozie.service.ShareLibService.mapping.file --> This needs to be configured in oozie-site.xml. need service restart, I would suggest go with option 1 OR 2
Hope this information helps.
Please accept the answer if you got required information :)
Created 07-01-2016 08:03 PM
@Kuldeep Kulkarni I understand these options. maybe I have not added my requirement explicitly. Oozie server uses following sharlib by default for respective Action Name. /user/oozie/share/lib/lib_$epochtime/$ActionName
where $epochtime -> latest timestamp upon reboot. How do we change this default location from Oozie server side to be more deterministic manner, ($epochtime is variable). Ok to restart Oozie server when there is a change, seamless to the user submitting the job. (dont want user to update their job.properties or workflow directories)