question Re: please help compile in Support Questions

please help compile

aliyesami — Thu, 06 Jul 2017 01:01:42 GMT

I posted this question before but got no response .

I am unable to compile this java code even though I am specifiying the right jars . please advise

[hdfs@hadoop1 ~]$ HADOOP_CLASSPATH=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-0.b15.el6_8.x86_64/lib/tools.jar
[hdfs@hadoop1 ~]$ export HADOOP_CLASSPATH
[hdfs@hadoop1 ~]$ javac -cp $HADOOP_CLASSPATH:/usr/hdp/2.5.3.0-37/hive2/lib/*  HiveAlterRenameTo.java
HiveAlterRenameTo.java:13: error: unreported exception ClassNotFoundException; must be caught or declared to be thrown
      Class.forName(driverName);
                   ^
1 error
[hdfs@hadoop1 ~]$ more HiveAlterRenameTo.java
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
public class HiveAlterRenameTo {
   private static String driverName = "org.apache.hive.jdbc.HiveDriver";
   public static void main(String[] args) throws SQLException {
      // Register driver and create driver instance
      Class.forName(driverName);
      // get connection
      Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");
      // create statement
      Statement stmt = con.createStatement();
      // execute statement
      stmt.executeQuery("ALTER TABLE PA_LANE_TXN_ORC RENAME TO PA_LANE_TXN;");
      System.out.println("Table Renamed Successfully");
      con.close();
   }
}

Re: please help compile

jsensharma — Thu, 06 Jul 2017 01:04:42 GMT

@Sami Ahmad

Please replace the :

 public static void main(String[] args) throws SQLException {

With the following:

 public static void main(String[] args) throws Exception {

OR as an alternative approach either use "try{ } catch(ClassNotFoundException cnef)".

OR try

 public static void main(String[] args) throws SQLException, ClassNotFoundException {

Re: please help compile

aliyesami — Thu, 06 Jul 2017 01:11:39 GMT

thanks !

I did the first change you asked and it compiled but now its failing when I run it . I changed "hive" to "hive2" but getting the error :

[hdfs@hadoop1 ~]$ grep hive2 HiveAlterRenameTo.java
      Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "", "");
[hdfs@hadoop1 ~

[hdfs@hadoop1 ~]$ java -cp .:$HADOOP_CLASSPATH:/usr/hdp/2.5.3.0-37/hive2/lib/*  HiveAlterRenameTo
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:418)
        at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:438)
        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:225)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:182)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at HiveAlterRenameTo.main(HiveAlterRenameTo.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 8 more

Re: please help compile

jsensharma — Thu, 06 Jul 2017 01:25:25 GMT

@Sami Ahmad

You have not set the CLASSPATH properly to inclide the required HIve and Hadoop JARs to it.

If you really want to run a simple hive client code then please refer to the following Example where i developed a very simple Java Based Hive Client using Maven:

https://github.com/jaysensharma/MiddlewareMagicDemos/tree/master/HDP_Ambari/Hive/HiveJavaClient

Re: please help compile

aliyesami — Thu, 06 Jul 2017 01:51:02 GMT

ok I tried including all the hive2 and hadoop jar directories and it fixed the issue but now I am getting connection refused , please advise

$ HADOOP_CLASSPATH=/usr/hdp/2.5.3.0-37/hive2/lib/*:/usr/hdp/2.5.3.0-37/hadoop/*:/usr/hdp/2.5.3.0-37/hadoop-mapreduce/*
$ export HADOOP_CLASSPATH
[hdfs@hadoop1 ~]$ java -cp .:$HADOOP_CLASSPATH:/usr/hdp/2.5.3.0-37/*  HiveAlterRenameTo
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default: java.net.ConnectException: Connection refused (Connection refused)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:209)

Re: please help compile

jsensharma — Thu, 06 Jul 2017 01:55:08 GMT

@Sami Ahmad

Change the line in your code that you posted Initially.

DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");

With the following line in your code : (notice "jdbc:hive" is replaced with "jdbc:hive2")

DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "", "");

The compile and run it as following:

export CLASSPATH=/usr/hdp/current/hive-client/lib/*:/usr/hdp/current/hive-client/jdbc/*:.:
javac HiveAlterRenameTo.java 
java HiveAlterRenameTo

If you want specific JAR names then copy the jars present in the above two directories and then manually add them in your classpath.

Re: please help compile

jsensharma — Thu, 06 Jul 2017 01:56:28 GMT

@Sami Ahmad

For different issues , Better to open separate HCC thread. That way we can manage the threads properly and users can quickly find one issue one resolution.

Re: please help compile

aliyesami — Thu, 06 Jul 2017 02:05:14 GMT

still getting connection refused , I also tried port 10001 but same result. my hive2 server is running on hadoop2 so I changed the line as below . I tried both ports 10000 and 10001 with same results.

 // get connection
      Connection con = DriverManager.getConnection("jdbc:hive2://hadoop2:10001/default", "", "");

Re: please help compile

jsensharma — Thu, 06 Jul 2017 02:07:09 GMT

@Sami Ahmad

Your Compilation Issue is resolved. For which the thread was opened.

Regarding Connection refused, the troubleshooting is completely different. Please open separate HCC thread.

And in Parallel check if the 10000 port is opened by the hive process on the mentioned host "localhost" or not? Or check the hostname is pointing to correct hive instance?

# netstat -tnlpa | grep 10000

Also check if there is any firewall issue in accessing the hive 10000 port if hive is running on remote host?

# telnet  localhost  1000
# telnet  $HIVE_HOSTNAME  10000