MR jobs calling 3rd party libs

[#1]

I want to submit a job to Yarn, which has dependencies to third party libraries. Map reduce jobs fail as they can't load these third party libraries on remote Hadoop nodes. Can I use Yarn in someway, to submit a whole package (including 3rd party libs), so MR jobs can successfully load them?

[#2 · Accepted solution]

did you try packaging all the 3rd party jars into uber jar and submit to cluster.

[#3]

@Koosha Tahmasebipour

you can leverage DistributedCache to upload your third party jars to HDFS https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/filecache/DistributedCache.html for MapReduce