Reply
Contributor
Posts: 47
Registered: ‎07-27-2015

Hive2 action sometimes exit(2)

HI,

we run hive2 action with oozie on cdh5.7.1 kerberos cluster.  

There is a strange behavior:

The below is my env:
5 node

6G hiveserver

8G hivemetastore  and are HA

we run hive2 action, sometimes the hive2 action failed.

The below is failed logs.

Error: Could not open client transport with JDBC Uri: jdbc:hive2://arch-od-tracker04.beta1.fn:10000/: Peer indicated failure: DIGEST-MD5: IO error acquiring password (state=08S01,code=0)
No current connection
Connected to: Apache Hive (version 1.1.0-cdh5.7.1)
Driver: Hive JDBC (version 1.1.0-cdh5.7.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Closing: 0: jdbc:hive2://arch-od-tracker04.beta1.fn:10000/
Aug 11, 2016 1:30:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
Intercepting System.exit(2)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]

 

 

0: jdbc:hive2://arch-od-tracker04.beta1.fn:10 (closed)>  set mapred.job.queue.na 
me=plarch;

********the top is the key, what happen? why the beeline cannot read the below sentence ****
<<< Invocation of Beeline command completed <<<

 Hadoop Job IDs executed by Beeline: 

Intercepting System.exit(2)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]

 

The below is successed logs:

 

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Connecting to jdbc:hive2://arch-od-tracker04.beta1.fn:10000/
Connected to: Apache Hive (version 1.1.0-cdh5.7.1)
Driver: Hive JDBC (version 1.1.0-cdh5.7.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
No rows affected (0.089 seconds)
INFO  : Compiling command(queryId=hive_20160811153737_7232c1fa-122e-4b47-b58a-30b06c69b8cd): use arch_onedata
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20160811153737_7232c1fa-122e-4b47-b58a-30b06c69b8cd); Time taken: 0.114 seconds
INFO  : Executing command(queryId=hive_20160811153737_7232c1fa-122e-4b47-b58a-30b06c69b8cd): use arch_onedata
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20160811153737_7232c1fa-122e-4b47-b58a-30b06c69b8cd); Time taken: 0.016 seconds
INFO  : OK
No rows affected (0.142 seconds)
INFO  : Compiling command(queryId=hive_20160811153737_d0ae1042-d934-4a69-bb88-b79895cdd8f8): ALTER TABLE odl_mem_modify_phone_fdt ADD PARTITION(ds='2016-08-10')
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20160811153737_d0ae1042-d934-4a69-bb88-b79895cdd8f8); Time taken: 0.111 seconds
INFO  : Executing command(queryId=hive_20160811153737_d0ae1042-d934-4a69-bb88-b79895cdd8f8): ALTER TABLE odl_mem_modify_phone_fdt ADD PARTITION(ds='2016-08-10')
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20160811153737_d0ae1042-d934-4a69-bb88-b79895cdd8f8); Time taken: 0.062 seconds
INFO  : OK
No rows affected (0.203 seconds)
Closing: 0: jdbc:hive2://arch-od-tracker04.beta1.fn:10000/
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).

 

=================================================================

>>> Invoking Beeline command line now >>>

0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> set mapred.job.queue.name=plarch; 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> use ${database_name};
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> ALTER TABLE ${table_name} ADD PAR 
TITION(ds='${dateStr}');
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 
0: jdbc:hive2://arch-od-tracker04.beta1.fn:10> 

****the top is the different than the failed logs, there is the beeline readed the all sentences** <<< Invocation of Beeline command completed <<< Hadoop Job IDs executed by Beeline: <<< Invocation of Main class completed <<< Oozie Launcher, capturing output data: ======================= #

 

 

What cause the issue? How to resolve it ?

Who can give me any idea?

Thanks in advance!

BR

Paul

Announcements