Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Query Fails in Tez, Runs in MR mode

Solved Go to solution
Highlighted

Hive Query Fails in Tez, Runs in MR mode

Hive Query fails with the below error when running in Tez Mode. My application dont have option to alter the execution mode. So tez is the only option. Is this a bug from Tez ? Query run fine in MR mode when tried from shell..

Error:

-------------------------------------------------------------------------------- Map 1 ..... RUNNING 494 278 0 216 343 0 Reducer 2 INITED 62 0 0 62 0 0 -------------------------------------------------------------------------------- VERTICES: 00/02 [=============>>-------------] 50% ELAPSED TIME: 41.84 s -------------------------------------------------------------------------------- Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1537195062060_938252_1_00, diagnostics=[Task failed, taskId=task_1537195062060_938252_1_00_000367, diagn=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOEon: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritable is not class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritablet class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:71) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false

View solution in original post

4 REPLIES 4
Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

Same problem here while triying to insert in the Sandbox HDP 2.6.4.

And the MapReduce mode is very slow too, it would be great be able to use Hive with TEZ.

Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

Mentor

@Krystle Salazar

Set that on the hive CLI

set hive.executio.engine=tez;

Should do

Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false

View solution in original post

Highlighted

Re: Hive Query Fails in Tez, Runs in MR mode

Worked like a charm!. Thank you.

Don't have an account?
Coming from Hortonworks? Activate your account here