Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Query Fails in Tez, Runs in MR mode

Solved Go to solution

Hive Query Fails in Tez, Runs in MR mode

New Contributor

Hive Query fails with the below error when running in Tez Mode. My application dont have option to alter the execution mode. So tez is the only option. Is this a bug from Tez ? Query run fine in MR mode when tried from shell..

Error:

-------------------------------------------------------------------------------- Map 1 ..... RUNNING 494 278 0 216 343 0 Reducer 2 INITED 62 0 0 62 0 0 -------------------------------------------------------------------------------- VERTICES: 00/02 [=============>>-------------] 50% ELAPSED TIME: 41.84 s -------------------------------------------------------------------------------- Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1537195062060_938252_1_00, diagnostics=[Task failed, taskId=task_1537195062060_938252_1_00_000367, diagn=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOEon: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritable is not class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritablet class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:71) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive Query Fails in Tez, Runs in MR mode

Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false
4 REPLIES 4

Re: Hive Query Fails in Tez, Runs in MR mode

New Contributor

Same problem here while triying to insert in the Sandbox HDP 2.6.4.

And the MapReduce mode is very slow too, it would be great be able to use Hive with TEZ.

Re: Hive Query Fails in Tez, Runs in MR mode

Mentor

@Krystle Salazar

Set that on the hive CLI

set hive.executio.engine=tez;

Should do

Re: Hive Query Fails in Tez, Runs in MR mode

Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false
Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

New Contributor

Worked like a charm!. Thank you.

Don't have an account?
Coming from Hortonworks? Activate your account here