Location of pig libraries

[#1]

On HDP I am looking for where parquet-pig-1.4.1.jar is located. I checked /usr/hdp/current/pig-client/lib and did not find anything.

Also where are 3rd party or custom libraries stored for pig? is it /usr/hdp/current/pig-client/lib ?

[#2 · Accepted solution]

@Sunile Manjee you need to download the jars for third party libs, they're not shipped with HDP by default. So in case of Parquet, download it from their site and place in /usr/hdp/current/pig-client/lib. You might actually have to build Parquet from source. https://github.com/Parquet/parquet-mr

[#3]

@Sunile Manjee just to clarify, you can place the jars anywhere but it's most convenient in the lib folder as Pig will see them. You can also take a look at this example https://community.hortonworks.com/questions/16415/pig-erro-execution.html#comment-16550

[#4]

@Artem Ervits I assume so but you confirmed my understanding.

[#5]

@Sunile Manjee

See this https://parquet.apache.org/downloads/

On internal note: Search in wiki and you will find lot of information on it.