Hive: Job running in-process (local Hadoop)

[#1]

Hi,

I am using Apache Hadoop 2.6.0 and Hive-1.1.1 with 3 node cluster..

I have few doubts related to a hive query. On submitting my Hive query, I am getting below response.

Doubts:

1. Why is it showing "Job running in-process (local Hadoop)"?

2. Why there is too many Stage-1 map = 100%?

3. Could you please suggest the reason for job failure?

Hive Query:

select *, dense_rank() over (partition by device_id order by total_bytes) from (select device_id,website,(upload_bytes+download_bytes) as total_bytes from CustWeb) a;

Response:

Job running in-process (local Hadoop)
2015-08-27 14:21:41,163 Stage-1 map = 0%,  reduce = 0%
2015-08-27 14:22:41,873 Stage-1 map = 0%,  reduce = 0%
2015-08-27 14:23:41,995 Stage-1 map = 0%,  reduce = 0%
2015-08-27 14:23:43,071 Stage-1 map = 1%,  reduce = 0%
2015-08-27 14:24:43,381 Stage-1 map = 1%,  reduce = 0%
2015-08-27 14:25:12,836 Stage-1 map = 2%,  reduce = 0%
2015-08-27 14:26:13,618 Stage-1 map = 100%,  reduce = 0%
2015-08-27 14:26:19,785 Stage-1 map = 2%,  reduce = 0%
2015-08-27 14:26:52,728 Stage-1 map = 3%,  reduce = 0%
2015-08-27 14:27:52,831 Stage-1 map = 3%,  reduce = 0%
2015-08-27 14:28:08,387 Stage-1 map = 4%,  reduce = 0%
2015-08-27 14:28:28,795 Stage-1 map = 5%,  reduce = 0%
2015-08-27 14:28:39,062 Stage-1 map = 100%,  reduce = 0%
2015-08-27 14:28:45,333 Stage-1 map = 5%,  reduce = 0%
2015-08-27 14:29:21,673 Stage-1 map = 6%,  reduce = 0%
2015-08-27 14:30:22,057 Stage-1 map = 6%,  reduce = 0%
2015-08-27 14:31:03,562 Stage-1 map = 7%,  reduce = 0%
2015-08-27 14:32:03,662 Stage-1 map = 7%,  reduce = 0%
2015-08-27 14:32:08,809 Stage-1 map = 100%,  reduce = 0%
2015-08-27 14:32:14,446 Stage-1 map = 7%,  reduce = 0%
2015-08-27 14:32:59,664 Stage-1 map = 8%,  reduce = 0%

.

.

.

2015-08-27 15:44:41,111 Stage-1 map = 71%,  reduce = 0%
2015-08-27 15:45:41,177 Stage-1 map = 71%,  reduce = 0%
2015-08-27 15:45:56,788 Stage-1 map = 72%,  reduce = 0%
2015-08-27 15:46:56,810 Stage-1 map = 72%,  reduce = 0%
2015-08-27 15:47:57,131 Stage-1 map = 72%,  reduce = 0%
2015-08-27 15:48:01,243 Stage-1 map = 73%,  reduce = 0%
2015-08-27 15:49:01,287 Stage-1 map = 73%,  reduce = 0%
Ended Job = job_local1266727216_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:8080/
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1:  HDFS Read: 184115396608 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

[#2]

Hi,

Could you please advice what could be the issue and the solution?

[#3]

I think you didnt the register the external Jar file that you are used on Creating the Table. Some times you will get message like 

"FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

MapReduce Jobs Launched: " when required jar file is not registered. 

[#4]

Hi Ramesh,

I didn't use any jar to create the Hive Table. I created the table by executing Hive query on Hive CLI.

The error message is:

2015-08-27 15:48:01,243 Stage-1 map = 73%,  reduce = 0%
2015-08-27 15:49:01,287 Stage-1 map = 73%,  reduce = 0%
Ended Job = job_local1266727216_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:8080/
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1:  HDFS Read: 184115396608 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec 

[#5]

Looks like you are running the job/command in local mode.

Try changing these properties in the beeline.

set mapreduce.framework.name;

set mapreduce.framework.name=yarn;

set mapreduce.framework.name;