I created a spark application that is configured thanks to a bunch of properties files that I specify at runtime with --files option of spark-submit command.
These local files are automatically copied to spark containers so that my job running in executors can read them to adjust its behavior.
Great, this works like a charm.
Now, I want to schedule this spark-submit action every hour with oozie, but couldn't find how to proceed to pass these configuration files properly to my spark job thanks to oozie...
I guess I have to copy these files to HDFS and ask oozie to launch the spark action and pass it thoses hdfs files, but cannot figure out how to achieve this...
Does anyone have a clue about this ?
Thanks a lot for your help