Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B

avatar
Visitor

hi guys

I got a problem with hive on tez

It's not always affect

But it really boring me

I get some information from my oozie ,like this

10074-error-msg.png

More detail as follows,thanks for your reply

Map 1: 0(+2,-1)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+2,-2)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+2,-3)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-3)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-4)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-5)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-6)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-7)/10 Map 3: 0/10 Reducer 2: 0/2 
Map 1: 0(+3,-8)/10 Map 3: 0/10 Reducer 2: 0/2 
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1480471191695_11172_1_00, diagnostics=[Task failed, taskId=task_1480471191695_11172_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 2 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 3 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:9, Vertex vertex_1480471191695_11172_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
Vertex killed, vertexName=Map 3, vertexId=vertex_1480471191695_11172_1_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:10, Vertex vertex_1480471191695_11172_1_02 [Map 3] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1480471191695_11172_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1480471191695_11172_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:2
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1480471191695_11172_1_00, diagnostics=[Task failed, taskId=task_1480471191695_11172_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 2 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 3 failed, info=[Error: Fatal Error cause TezChild exit.:java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
 at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:412)
 at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:301)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor$1.call(MapRecordProcessor.java:109)
 at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
 at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.<init>(MapRecordProcessor.java:106)
 at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:132)
 at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
 at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
 at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:9, Vertex vertex_1480471191695_11172_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Map 3, vertexId=vertex_1480471191695_11172_1_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:10, Vertex vertex_1480471191695_11172_1_02 [Map 3] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1480471191695_11172_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1480471191695_11172_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:2
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
1 ACCEPTED SOLUTION

avatar
Not applicable

- Additionally, as a side note the following JARs contains the "org.apache.commons.codec.binary.Base64" class :

/usr/hdp/current/oozie-client/share/lib/hive2/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/hive/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/share/lib/hive/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/pig/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/spark/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/libserver/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/libserver/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/oozie-server/webapps/oozie/WEB-INF/lib/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/oozie-server/webapps/oozie/WEB-INF/lib/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/libtools/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/lib/commons-codec-1.4.jar


/usr/hdp/current/tez-client/lib/commons-codec-1.4.jar

.

View solution in original post

3 REPLIES 3

avatar
Not applicable

@苏 斌

The error which you are getting indicates that you are using an incorrect version of "commons-codec-*.jar" in your CLASSPATH.

Can you please check & share the classpath "java.class.path" value from the below command and share it with us:

$JAVA_HOME/bin/jinfo  $PID  | grep java.class.path 

Example:

/usr/jdk64/jdk1.8.0_60/bin/jinfo 24619   | grep java.class.path

.

Are you using correct version of "commons-codec" jar ?

Example:

/usr/hdp/current/tez-client/lib/commons-codec-1.4.jar

.

The "decodeBase64(java.lang.String)" with string as argument was added to "Base64.java" since version 1.4

https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html#...

avatar
Visitor
@jss

hi, thanks for your detailed reply and i was sorry it's a long time

it's a long week for me

I use oozie to drive my hive on tez job via hiveserver2 to have this issue.

Afterwards I change hive execution engine to mr, the job goes well, doesn't not occur this issue again, but job finished so slowly

So i think it's about tez lib

It's hard to use $JAVA_HOME/bin/jinfo $PID | grep java.class.path to get the hive runtime classpath

I pick up some information from oozie console

HADOOP_CLASSPATH=::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/2.3.2.0-2950/tez/*:/usr/hdp/2.3.2.0-2950/tez/lib/*:/usr/hdp/2.3.2.0-2950/tez/conf::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/2.3.2.0-2950/tez/*:/usr/hdp/2.3.2.0-2950/tez/lib/*:/usr/hdp/2.3.2.0-2950/tez/conf:

and uploaded my classpath info as an attachment

hope this can provide u more information to help me solve this issue

I appreciate it, thank youclasspath.txt

avatar
Not applicable

- Additionally, as a side note the following JARs contains the "org.apache.commons.codec.binary.Base64" class :

/usr/hdp/current/oozie-client/share/lib/hive2/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/hive/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/share/lib/hive/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/pig/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/share/lib/spark/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/libserver/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/libserver/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/oozie-server/webapps/oozie/WEB-INF/lib/avatica-1.8.0.2.5.0.0-1133.jar
/usr/hdp/current/oozie-client/oozie-server/webapps/oozie/WEB-INF/lib/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/libtools/commons-codec-1.4.jar
/usr/hdp/current/oozie-client/lib/commons-codec-1.4.jar


/usr/hdp/current/tez-client/lib/commons-codec-1.4.jar

.