Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2 job in oozie


Spark2 job in oozie

I am trying to run a spark2 job in oozie and using HDP 2.6.0 and it fails with the following reason

java.lang.IllegalArgumentException: Attempt to add (hdfs:// multiple times to the distributed cache.

and from whatever search i have made, the current oozie version supported in HDP 2.6 doesnt support spark2 job.
IS there a workaround to make it work?

Any thoughts would be great

Re: Spark2 job in oozie


you need remove duplicate jars from ../spark2/ directory

follow this:

Don't have an account?
Coming from Hortonworks? Activate your account here