Hi, I need to run spark job from oozie java action as regular java application with main class.
I can't use oozie spark or shell actino for many reasons.
I've built jar with spark-yarn 2.2 hdp 188.8.131.52 dependecy. I can easilly run it using spark-submit.
I try to run same jar using same main class as oozie java action and get exception:
Caused by: java.lang.IllegalStateException: Library directory '/hadoop/yarn/local/usercache/root/appcache/application_1521375636129_0022/container_e06_1521375636129_0022_01_000002/assembly/target/scala-2.11/jars' does not exist; make sure Spark is built.
I took a look at the source code, there are some weird checks.... What is the right way to disable them?