Reply
New Contributor
Posts: 8
Registered: ‎06-09-2017

Hive JDBC Multithread error

[ Edited ]

Hi

 

I am using HiveJDBC41(2.5.18) driver to fetch data from Hive database and i have some strange error. I have multithread application which load data from Hive. In my case i have 3 threads, each thread open his own hive connection and execute sql query into ResultSet, but ResultSet return unexpected result, it's mean that in one thread i see result of another thread and didn't see result of it self. For example:

1 thread - SELECT * FROM TAB1

2 thread - SELECT * FROM TAB2

3 thread - SELECT * FROM TAB3

 

When i run my application i see that 1-thread execute SELECT * FROM TAB1(correct), second execute (SELECT * FROM TAB2) third execute SELECT * FROM TAB3, but in ResultSet in 3 thread i see result of 1-thread(SELECT * FROM TAB1).

 

Some times it's works fine it happened when threads started with waiting. Could, please anybody help me in this stranged situation?

 

P.S. When i start my application with one thread all works fine.

Cloudera Employee
Posts: 16
Registered: ‎01-07-2014

Re: Hive JDBC Multithread error

Hi Dobroff,

 

What CDH version are you using ?

 

Do you see any exceptions in HiveServer2 logs ?

 

thanks.

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

Hi

 

I unfortunately i don´t have access to logs. What you mean by CHD? Hive version? My hive version is 1.2

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

Hi

 

I see that it problem occurs when two different threads open Statement in same time and loop on ResultSet see just last Result.It is really strange because i open my JDBC-Connection in each thread and don't use static methods or variables.... May be JDBC driver has some options to allow multi-threading?

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

Looks like it's problem with Metadata object not in ResultSet object

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

Any ideas?

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

hadoop version returns

 

Hadoop 2.6.0-cdh5.8.3

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

I debug my code and that's what i found.

 

I have table with two columns (SOME_COLUMN1, SOME_COLUMN2), and data 1,2 

 

I have code like this:

sqlStmt = "SELECT SOME_COLUM<thread_id> FROM TABLE";

ResultSet rs = this.BDconnection.createStatement().executeQuery(sqlStmt);

 

while(rs.next())

{

    rs.getString(1);

    rs.getString("SOME_COLUMN1<thread_id>");

}

 

For example thread1 execute "SELECT SOME_COLUMN1 FROM TABLE" and thread2 execute "SELECT SOME_COLUMN2 FROM TABLE" and get SOME_COLUMN1 and SOME_COLUMN2 in getString as well. If this two threads run in the same time the first started thread occurs a error what SOME_COLUMN2 not exists but if i do rs.getString(1) data was like in SOME_COLUMN1(in my case it's "1"), for my opion it's mean what data in ResultSet is correct but MetaData for this ResultSet is bad.

New Contributor
Posts: 8
Registered: ‎06-09-2017

Re: Hive JDBC Multithread error

Still no ideas?

Champion
Posts: 597
Registered: ‎05-16-2016

Re: Hive JDBC Multithread error

HiveServer2 Log  could be helpful to dig more about the issue .

since it act as a Session facade

Announcements