question Lab 3 - Pig Risk Factor Analysis: Application remains in running status in Archives of Support Questions (Read Only)

Lab 3 - Pig Risk Factor Analysis: Application remains in running status

Eukrev — Sat, 20 Feb 2016 20:36:16 GMT

Hi,

Could you help me to run a pig script with successful status.

As mentioned in the Lab 3 exercise, geolocation table was created in hive and the pig script was created with only load transformation.

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();

DUMP a LIMIT 1;

Tried executing the script(not on Tez) arguments with -useHCatalog.

In the ResourceManagerUI the status show succeeded with Cantainers allocated 2. But, in the PIGUI the status is still RUNNING. It neither errors or return a result tab.

What might be the issue? How to check if the metadata exist in HCatalog? WebHCat server is in started status. Where should I look for Heap memory?

Thanks In advance for the support.

resourcemanagerui.png

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

nsabharwal — Sat, 20 Feb 2016 20:48:29 GMT

@Revathy Mourouguessane

1) Check MapReduce --> what are those alerts? Make sure that HDFS, Yarn and MapReduce services are up

2) It looks like your pig job did finish but check the pig view log

3)hive.png - You can see geolocation table , thats your hive table

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

nsabharwal — Sun, 18 Aug 2019 12:58:59 GMT

@Revathy Mourouguessane heap memory

You can click HDFS and in filter type heap

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

aervits — Sat, 20 Feb 2016 21:40:21 GMT

Make sure mapreduce is up if youre not executing using Tez. It is preferred if you use tez though as it is faster.

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

Eukrev — Sun, 21 Feb 2016 10:19:22 GMT

Thank you.

I have tried with MapReduce2 and Tez2. The MapReduce2 is set to green. The heap size is same as in the attachement.

job-1456016544519-0006-logs.txt

job-1456016544519-0007-logs.txt

It errors with Could not resolve org.apache.hcatalog.pig.HCatLoader. In the arguments -useHCatalog is set.

ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.] Failed to parse: Pig script failed to parse: <file script.pig, line 1, column 29> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.] at org.apache.pig.parser.QueryParser

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

Eukrev — Sun, 21 Feb 2016 10:20:11 GMT

I tried both and get org.apache.hcatalog.pig.HCatLoader. The arguments of script is set as -useHCatalog. Thanks!!!

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

aervits — Sun, 21 Feb 2016 10:29:30 GMT

Package is incorrect its USING org.apache.hive.hcatalog.pig.HCatLoader();

It was fixed in future release of sandbox use tez and make sure to use the HcatLoader I specified.

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

nsabharwal — Sun, 21 Feb 2016 10:55:27 GMT

@Revathy Mourouguessane Please see this

http://stackoverflow.com/questions/29990236/flag-usehcatalog-not-working

See this

I was facing the issue in HDP 2.3 and Pig 0.15 .

Package name for HCatLoader() class is different in Hortonworks distribution.

The following worked for me

USING org.apache.hive.hcatalog.pig.HCatLoader()

instead of USING org.apache.hcatalog.pig.HCatLoader();

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

Eukrev — Sun, 21 Feb 2016 15:07:13 GMT

Thank you.

It is working fine now.

Re: Lab 3 - Pig Risk Factor Analysis: Application remains in running status

Eukrev — Sun, 21 Feb 2016 15:09:02 GMT

Thanks. Its working fine now.