question Re: Location of pig libraries in Archives of Support Questions (Read Only)

Location of pig libraries

sunile_manjee — Fri, 26 Feb 2016 00:45:07 GMT

On HDP I am looking for where parquet-pig-1.4.1.jar is located. I checked /usr/hdp/current/pig-client/lib and did not find anything.

Also where are 3rd party or custom libraries stored for pig? is it /usr/hdp/current/pig-client/lib ?

Re: Location of pig libraries

aervits — Fri, 26 Feb 2016 00:46:54 GMT

@Sunile Manjee you need to download the jars for third party libs, they're not shipped with HDP by default. So in case of Parquet, download it from their site and place in /usr/hdp/current/pig-client/lib. You might actually have to build Parquet from source. https://github.com/Parquet/parquet-mr

Re: Location of pig libraries

aervits — Fri, 26 Feb 2016 01:00:36 GMT

@Sunile Manjee just to clarify, you can place the jars anywhere but it's most convenient in the lib folder as Pig will see them. You can also take a look at this example https://community.hortonworks.com/questions/16415/pig-erro-execution.html#comment-16550

Re: Location of pig libraries

sunile_manjee — Fri, 26 Feb 2016 01:02:54 GMT

@Artem Ervits I assume so but you confirmed my understanding.

Re: Location of pig libraries

nsabharwal — Fri, 26 Feb 2016 01:21:46 GMT

@Sunile Manjee

See this https://parquet.apache.org/downloads/

On internal note: Search in wiki and you will find lot of information on it.