Hive org.apache.hadoop.hive.ql.exec.tez.TezTask exception

[#1]

Hi,

I´m using the vm´s of ambari and installed a cluster of 5 nodes (c7001 - c7005). I imported a csv file into HBase and mapped a Hive table to it. I want to write a Java programm which executes some simple queries like count, read, filters, group by..

I get the following exception:

[hive@c7002 vagrant]$ java -cp SparkThesisTest-4.3.0-SNAPSHOT-jar-with-dependencies.jar thesis.test.sparkthesistest.hbase.read.HiveRead
log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.Utils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
        at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:348)
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:251)
        at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:434)
        at thesis.test.sparkthesistest.hbase.read.HiveRead.main(HiveRead.java:35)

The Java Code is simple and looks like:

      // Register driver and create driver instance
      Class.forName("org.apache.hive.jdbc.HiveDriver");
      // get connection
      Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive", ""); 
      // create statement
      Statement stmt = con.createStatement();      
      // execute statement
      stmt.executeQuery("INSERT OVERWRITE DIRECTORY '/tmp/hive_read_countries.csv' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' select distinct City, Country from hbase_table_climate where country like \"Germany\"");
      con.close();

Im executing the Java Application on c7002. This is where the HiveServer2 runs.

Im using maven and installed the following dependency:

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>2.1.0</version>
        </dependency>

If execute the exact same query through the "Hive View" or inside the "Hive Shell" it works

INSERT OVERWRITE DIRECTORY '/tmp/hive_read_countries.csv' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' select Country from hbase_table_climate

[#2]

I just found that if I adjust the code like the following, the execution works:

        // execute statement
        ResultSet res = stmt.executeQuery("select Country from hbase_table_climate");
        while (res.next()) {
            System.out.println(res.getString(1));
         }

If I adjust a little bit more complex select statement it crashes with the mentioned exception:

        ResultSet res = stmt.executeQuery("select distinct City, Country from hbase_table_climate where country like \"Germany\"");

[#3]

@Morten R., It is difficult to get to the root cause from the error message that you have provided. Can you share detailed error message from the hiveserver2 log and application log? You can generate application log by running 'yarn logs -applicationId <application_id>'.

[#4]

@Morten R.

What do you see in HS2 logs and also the application logs? Can you change the log level to DEBUG and then try to get the error stacktrace?

[#5 · Accepted solution]

I configured the execution engine to MapReduce instead of Tez, this works for me. Since I´m only testing and learning some stuff its ok for me.. (I did not know hive got several engines)