question Re: pig -useHCatalog not loading all the jars required in Archives of Support Questions (Read Only)

pig -useHCatalog not loading all the jars required

bhaskaran_peria — Sat, 26 Dec 2015 21:07:01 GMT

In latest practice exam in AWS, pig -useHCatalog not loading all the jars required. I am trying to fix these by adding classpath or using REGISTER. My question here is do we need to deal/spend time with such environmental issues in real exam?. Other day when I started AWS instance, services were not running at first instance. @rich helped me to address this.

Re: pig -useHCatalog not loading all the jars required

rich1 — Sun, 27 Dec 2015 00:46:50 GMT

Can you share the error? I am guessing it is either not an error (there is a warning you can ignore), or you are using the wrong package name for HCatLoader. The class name to use is:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if you get it to work.

Re: pig -useHCatalog not loading all the jars required

aervits — Sun, 27 Dec 2015 12:15:44 GMT

are you experiencing this? https://community.hortonworks.com/questions/2346/bug-in-hdp-232-and-earlier-releases-ambari-pig-vie.html

Re: pig -useHCatalog not loading all the jars required

bsuresh — Sun, 27 Dec 2015 21:54:26 GMT

As @rich said, We need to use

org.apache.hive.hcatalog.pig.HCatLoader

We should not use

org.apache.hcatalog.pig.HCatLoader (We have this package in so many materials like Pig Programming )

Re: pig -useHCatalog not loading all the jars required

aervits — Sun, 27 Dec 2015 22:55:28 GMT

@Suresh Bonam

it was fixed in Ambari 2.2. When new Sandbox is released, it will reflect the fix in that version.

Re: pig -useHCatalog not loading all the jars required

bsuresh — Mon, 28 Dec 2015 00:23:45 GMT

@Artem Ervits

Thank you.Really we don't about this fix.We encountered above mentioned problem in Ambari 1.7 with HDP 2.2. I guess,HDP certification exam is provided with HDP 2.2. So i think ,we can get above mentioned problem.

Re: pig -useHCatalog not loading all the jars required

rich1 — Mon, 28 Dec 2015 10:11:20 GMT

That issue with Ambari has nothing to do with the environment of the practice exam. The package name for HCatLoader and HCatStorer changed in HDP 2.0. Using the proper class name (shown above) is the answer to this question.

Re: pig -useHCatalog not loading all the jars required

aervits — Mon, 28 Dec 2015 10:20:04 GMT

even though those classes have been moved into org.apache.hive it was still pointing to wrong package names in Ambari Pig View until 2.2. I'm not familiar with the practice exam environment, I know that current Sandbox has the same issue and I'm not competing for the best answer @rich.

Re: pig -useHCatalog not loading all the jars required

rich1 — Mon, 28 Dec 2015 10:27:26 GMT

No worries. I am just trying to make it clear for anyone in the future who is preparing for our exams. The HCatLoader and HCatStorer classes have different package names, so the subject of this post is misleading and has nothing to do with missing JAR files. If you use the correct class name then everything works fine.

Re: pig -useHCatalog not loading all the jars required

bhaskaran_peria — Wed, 30 Dec 2015 06:06:08 GMT

Apologies for inappropriate subject wordings. It is the package name as mentioned by @rich

Re: pig -useHCatalog not loading all the jars required

ramesh_k_egamam — Thu, 31 Dec 2015 06:43:15 GMT

I have used org.apache.hive.hcatalog.pig.HCatLoader in practice exam instance still getting 3 errors for missing jar files. How to fix to continue to work on that instance?

Re: pig -useHCatalog not loading all the jars required

rich1 — Fri, 01 Jan 2016 02:47:34 GMT

What errors are you getting? And what is the command you are using to run the code? Perhaps you could share your code also.

Thanks.

Re: pig -useHCatalog not loading all the jars required

dineshc — Tue, 10 Jan 2017 08:42:48 GMT

@bhaskaran periasamy

You must use the correct package in you Pig script.

data = LOAD 'db.table' using org.apache.hive.hcatalog.pig.HCatLoader();