how to add third party jars for a mapreduce program

prklearning — Tue, 19 Apr 2016 15:14:09 GMT

Hi All,

Am getting this error

cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.vertica.hadoop.VerticaOutputFormat not found
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.vertica.hadoop.VerticaOutputFormat not found

However , I have added the third party jar into the /usr/hdp/2.3.4.0-***/hadoop/lib

and have also used -libjar option while running the command .

can anyone let me know where to set the hadoop classpath correctly , and if we are copying the third party jars into the hadoop source folder there are many folder within like hadoop-hdfs,hadoop-mapreduce,hadoop-httpf,hadoop-yarn do we copy the third party jars here too ????

Re: how to add third party jars for a mapreduce program

aervits — Tue, 19 Apr 2016 16:22:51 GMT

You need to add jars with -libjars not -libjar

you can also add your connector jar as dependency to your project as I am not able to find a maven pom you can include. You also need to make sure if you do put the jar in lib of hadoop ditectories, you need to place it on every node.

You can also get familiar with DistributedCache API to upload jars in hdfs for access. https://hadoop.apache.org/docs/current/api/org/apache/hadoop/filecache/DistributedCache.html

question how to add third party jars for a mapreduce program in Archives of Support Questions (Read Only)

how to add third party jars for a mapreduce program

Re: how to add third party jars for a mapreduce program