question Re: Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception in Archives of Support Questions (Read Only)

Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

morten_riedel — Wed, 22 Mar 2017 20:03:42 GMT

Hi,

I´m using the vm´s of ambari and installed a cluster of 5 nodes (c7001 - c7005). I imported a csv file into HBase and mapped a Hive table to it. I want to write a Java programm which executes some simple queries like count, read, filters, group by..

I get the following exception:

[hive@c7002 vagrant]$ java -cp SparkThesisTest-4.3.0-SNAPSHOT-jar-with-dependencies.jar thesis.test.sparkthesistest.hbase.read.HiveRead
log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.Utils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
        at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:348)
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:251)
        at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:434)
        at thesis.test.sparkthesistest.hbase.read.HiveRead.main(HiveRead.java:35)

The Java Code is simple and looks like:

      // Register driver and create driver instance
      Class.forName("org.apache.hive.jdbc.HiveDriver");
      // get connection
      Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive", ""); 
      // create statement
      Statement stmt = con.createStatement();      
      // execute statement
      stmt.executeQuery("INSERT OVERWRITE DIRECTORY '/tmp/hive_read_countries.csv' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' select distinct City, Country from hbase_table_climate where country like \"Germany\"");
      con.close();

Im executing the Java Application on c7002. This is where the HiveServer2 runs.

Im using maven and installed the following dependency:

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>2.1.0</version>
        </dependency>

If execute the exact same query through the "Hive View" or inside the "Hive Shell" it works

INSERT OVERWRITE DIRECTORY '/tmp/hive_read_countries.csv' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' select Country from hbase_table_climate

Re: Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

morten_riedel — Wed, 22 Mar 2017 20:44:32 GMT

I just found that if I adjust the code like the following, the execution works:

        // execute statement
        ResultSet res = stmt.executeQuery("select Country from hbase_table_climate");
        while (res.next()) {
            System.out.println(res.getString(1));
         }

If I adjust a little bit more complex select statement it crashes with the mentioned exception:

        ResultSet res = stmt.executeQuery("select distinct City, Country from hbase_table_climate where country like \"Germany\"");

Re: Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

jvaria — Thu, 23 Mar 2017 03:47:16 GMT

@Morten R., It is difficult to get to the root cause from the error message that you have provided. Can you share detailed error message from the hiveserver2 log and application log? You can generate application log by running 'yarn logs -applicationId <application_id>'.

Re: Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

vgumashta — Thu, 23 Mar 2017 10:09:41 GMT

@Morten R.

What do you see in HS2 logs and also the application logs? Can you change the log level to DEBUG and then try to get the error stacktrace?

Re: Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

morten_riedel — Thu, 23 Mar 2017 22:19:10 GMT

I configured the execution engine to MapReduce instead of Tez, this works for me. Since I´m only testing and learning some stuff its ok for me.. (I did not know hive got several engines)