Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive Query Fails in Tez, Runs in MR mode

avatar

Hive Query fails with the below error when running in Tez Mode. My application dont have option to alter the execution mode. So tez is the only option. Is this a bug from Tez ? Query run fine in MR mode when tried from shell..

Error:

-------------------------------------------------------------------------------- Map 1 ..... RUNNING 494 278 0 216 343 0 Reducer 2 INITED 62 0 0 62 0 0 -------------------------------------------------------------------------------- VERTICES: 00/02 [=============>>-------------] 50% ELAPSED TIME: 41.84 s -------------------------------------------------------------------------------- Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1537195062060_938252_1_00, diagnostics=[Task failed, taskId=task_1537195062060_938252_1_00_000367, diagn=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOEon: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritable is not class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritablet class org.apache.hadoop.io.BytesWritable at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:71) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150) ... 14 more

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false

View solution in original post

4 REPLIES 4

avatar

Same problem here while triying to insert in the Sandbox HDP 2.6.4.

And the MapReduce mode is very slow too, it would be great be able to use Hive with TEZ.

avatar
Master Mentor

@Krystle Salazar

Set that on the hive CLI

set hive.executio.engine=tez;

Should do

avatar
Expert Contributor

Probably you might be hitting TEZ-2741.

can you try to run the query by setting below config

hive.compute.splits.in.am=false

avatar

Worked like a charm!. Thank you.