Hive job failed on Tez - Failed to execute tez graph

[#1]

Hi. I created Oozie workflow includes HDFS Fs, Sqoop and Hive jobs. The first two jobs work great - Sqoop imports data from Oracle database and save to HDFS. But then there is a problem with Hive, more precisely with Tez. When I try to execute only one Hive statement there is no problem:

LOAD DATA INPATH '/user/apb_general/dms_update' OVERWRITE INTO TABLE DMS_TEST_MATGRA;

But when I add another statement:

LOAD DATA INPATH '/user/apb_general/dms_update' OVERWRITE INTO TABLE DMS_TEST_MATGRA;

INSERT OVERWRITE TABLE DMS_TEST_MATGRA_DIST SELECT DISTINCT macaddr, techchannelname, channelzapnumber FROM DMS_TEST_MATGRA;

job ends with error:

11938 [main] ERROR org.apache.hadoop.hive.ql.exec.Task  - Failed to execute tez graph.
java.lang.IllegalArgumentException: size of topologicalVertexStack is:3 while size of vertices is:2, make sure they are the same in order to sort the vertices

I found a ticket in JIRA which is associated with this error: DAG.createDag() does not clear local state on repeat calls

But fixed versions are 0.7.2 and newer. HDP provides Tez 0.7.0. Do you know how can I overcome this problem?

[#2 · Accepted solution]

This issue is fixed in HDP 2.5 that also comes with Tez 0.7.0. sometimes we backport critical fixes. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_release-notes/content/patch_tez.html

[#3]

Thank you for the information 🙂