how to setup oozie custom share lib for an action in places other then sharelib_timestamp and reference them with a name so then later we can use this in the configuration ?

[#1]

how to setup oozie custom share lib for an action in places other then sharelib_timestamp and reference them with a name so then later we can use this in the configuration ? We are trying to stage a shared lib directory outside of the normal /users/oozie/share/libs/XXXXXXXXX

Assumption is that oozie.service.ShareLibService.mapping.file is what will acccomplish this for us

e.g. I already have

/user/oozie/share/lib/lib_$epochtime/spark

But I want to create another directory for specific spark version and call use this as share lib for my spark action.

/user/oozie/share/lib/spark_161

[#2]

@Saumil Mayani

You can put your custom jars in /user/oozie/share/lib/spark_161 and add below line in your job.properties

to reference them

oozie.libpath=/user/oozie/share/lib/spark_161,another/location/if/you/want

[#3]

Hi @Kuldeep Kulkarni can this "oozie.libpath" be used in conjunction to oozie.use.system.libpath=true. Also, how can infrastructure team enforce this for a particular action without tenants needing to change their job.properties. How does "oozie.service.ShareLibService.mapping.file" work? can this be used instead?

[#4]

@Saumil Mayani

1. Yes you can use both in job.properties.

2. If you want it to be workflow specific then there is an alternative way

- No need to add anything in job.properties or sharelib. You can just create a directory named “lib” next to your workflow.xml in HDFS and put jars in there.

3. Regarding oozie.service.ShareLibService.mapping.file --> This needs to be configured in oozie-site.xml. need service restart, I would suggest go with option 1 OR 2

Hope this information helps.

Please accept the answer if you got required information :)

[#5]

@Kuldeep Kulkarni I understand these options. maybe I have not added my requirement explicitly. Oozie server uses following sharlib by default for respective Action Name. /user/oozie/share/lib/lib_$epochtime/$ActionName

where $epochtime -> latest timestamp upon reboot. How do we change this default location from Oozie server side to be more deterministic manner, ($epochtime is variable). Ok to restart Oozie server when there is a change, seamless to the user submitting the job. (dont want user to update their job.properties or workflow directories)

[#6]

@cnormile please advise