question MapReduce stuck, Pig, Hive in HDP not work in Support Questions

MapReduce stuck, Pig, Hive in HDP not work

orozcohsu — Mon, 19 Aug 2019 10:51:43 GMT

I follow Automation install in one VM with 16GB, 4 cores

1. After installed, all service are in green light. It's good.

2. When I use Pig view, I add New Script and write a simple code. The task is stuck, it seems not mapreduce not running.

3. I check the yarn log, it is always in RUNNING

4. I then try to test Hive in Upload Table, I choose one local file to upload to table hotels.

When I submit to upload, it always stop in this message

5. Then I go to hive to see table hotel I just uploaded. It is empty, nothing has been uploaded.

6. I think my HDP (mapreduce) is not working, only HDFS is fine.

Someone can tell me what's going on? I also try to change some yarn memory setting I google from internet posts.

Unfortunately it doesn't work.

Please help, thanks.

Re: MapReduce stuck, Pig, Hive in HDP not work

PranayV — Fri, 03 Jun 2016 00:09:23 GMT

To troubleshoot

1) Kill all existing running/waiting applications. On terminal type "Yarn application -list". From the list, for each application id listed do "yarn application -kill <applicationid>".

2) Go to Ambari UI and select mapreduce. on top right, service-> run service check. Ambari will automatically run test mapreduce job. if that fails post the log here.

3) Do the same for Hive service and Yarn service one service check at a time.

Re: MapReduce stuck, Pig, Hive in HDP not work

orozcohsu — Mon, 19 Aug 2019 10:51:19 GMT

Thanks,

Tez service check has stopped at 35% in error.

This is the Tez log (log.txt), thank you

Then I try to run a MapReduce Pig without Tez, I see its output. It is working.

I release if my running job is related to Tez, that will be failed.

I try a test for Hive loading hdfs file into Hive table, I found it uses Tez.

Re: MapReduce stuck, Pig, Hive in HDP not work

PranayV — Fri, 03 Jun 2016 03:03:14 GMT

Do you see anything in log files of Hive/Tez?

Go to Hive shell. Create test table and load some data to that table. See if Tez is invoked and query runs fine.

post the hive log console and Hive log file errors if it fails.

Re: MapReduce stuck, Pig, Hive in HDP not work

orozcohsu — Mon, 19 Aug 2019 10:51:11 GMT

I think my Tez is not working well.

here is my test (in Ambari)

1. Using Pig without Tez => (it is ok)

2. Using Pig with Tez => (stuck in mapreduce)

3. Using Hive join table (default using Tez) => (it is finished in Ambari, but RM UI is still in stuck)

Please see the application_1464887803963_0048

4. I use Pig on Tez in terminal (pig -x tez pig.test) => (it is ok)

5. I re-run the Tez service check, it's in green.

I don't know why? I think my Tez in Amari is not working, please help

Thanks

Re: MapReduce stuck, Pig, Hive in HDP not work

orozcohsu — Mon, 19 Aug 2019 10:50:58 GMT

Thanks, I solve the question.

The solution is to increase the core number for Ambari.

Tez and other service are working well.

Thank you!