Support Questions

cjervis · ‎09-09-2021

hi experts,

When trying to test tez, i get the following error.

[root@test01 ~]# hadoop --config /etc/hadoop/conf jar /usr/lib/tez/tez-examples*.jar orderedwordcount /tmp/tezsmokeinput/sample-tez-test /tmp/tezsmokeoutput/
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
21/09/09 11:23:56 INFO shim.HadoopShimsLoader: Trying to locate HadoopShimProvider for hadoopVersion=2.10.1, majorVersion=2, minorVersion=10
21/09/09 11:23:56 INFO shim.HadoopShimsLoader: Picked HadoopShim org.apache.tez.hadoop.shim.HadoopShim28, providerName=org.apache.tez.hadoop.shim.HadoopShim28Provider, overrideProviderViaConfig=null, hadoopVersion=2.10.1, majorVersion=2, minorVersion=10
21/09/09 11:23:57 INFO client.TezClient: Tez Client Version: [ component=tez-api, version=0.9.2, revision=81ad7b000cec0503b9a1d5521fdaf0129443b536, SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git, buildTime=2020-11-28T13:10:15Z ]
21/09/09 11:23:57 INFO client.RMProxy: Connecting to ResourceManager at test01.com/10.49.4.11:8050
21/09/09 11:23:57 INFO client.AHSProxy: Connecting to Application History server at test02.com/10.49.4.12:10200
21/09/09 11:23:57 INFO examples.OrderedWordCount: Running OrderedWordCount
21/09/09 11:23:57 INFO client.TezClient: Submitting DAG application with id: application_1629805664278_0020
21/09/09 11:23:57 INFO client.TezClientUtils: Using tez.lib.uris value from configuration: /bgtp/apps/1.0/tez/tez.tar.gz
21/09/09 11:23:57 INFO client.TezClientUtils: Using tez.lib.uris.classpath value from configuration: null
21/09/09 11:23:58 INFO client.TezClient: Tez system stage directory hdfs://test/tmp/root/staging/.tez/application_1629805664278_0020 doesn't exist and is created
21/09/09 11:23:58 INFO conf.Configuration: resource-types.xml not found
21/09/09 11:23:58 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
21/09/09 11:23:58 INFO resource.ResourceUtils: Adding resource type - name = memory-mb, units = Mi, type = COUNTABLE
21/09/09 11:23:58 INFO resource.ResourceUtils: Adding resource type - name = vcores, units = , type = COUNTABLE
21/09/09 11:23:58 INFO client.TezClient: Submitting DAG to YARN, applicationId=application_1629805664278_0020, dagName=OrderedWordCount, callerContext={ context=TezExamples, callerType=null, callerId=null }
21/09/09 11:23:58 INFO impl.YarnClientImpl: Submitted application application_1629805664278_0020
21/09/09 11:23:58 INFO client.TezClient: The url to track the Tez AM: http://test01.com:8088/proxy/application_1629805664278_0020/
21/09/09 11:23:59 INFO client.TezClient: App did not succeed. Diagnostics: Application application_1629805664278_0020 failed 2 times due to AM Container for appattempt_1629805664278_0020_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2021-09-09 11:23:59.041]Exception from container-launch.
Container id: container_1629805664278_0020_02_000001
Exit code: 1

[2021-09-09 11:23:59.042]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster


[2021-09-09 11:23:59.042]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster


For more detailed output, check the application tracking page: http://test01.com:8088/cluster/app/application_1629805664278_0020 Then click on links to logs of each attempt.
. Failing the application.
21/09/09 11:23:59 INFO client.DAGClientImpl: DAG completed. FinalState=FAILED
21/09/09 11:23:59 INFO examples.OrderedWordCount: DAG diagnostics: [Application application_1629805664278_0020 failed 2 times due to AM Container for appattempt_1629805664278_0020_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2021-09-09 11:23:59.041]Exception from container-launch.
Container id: container_1629805664278_0020_02_000001
Exit code: 1

[2021-09-09 11:23:59.042]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster


[2021-09-09 11:23:59.042]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster


For more detailed output, check the application tracking page: http://test01.com:8088/cluster/app/application_1629805664278_0020 Then click on links to logs of each attempt.
. Failing the application.]
[root@test01 ~]#

Any help is much appreciated.

Thanks,

ggangadharan · ‎11-21-2023

It seems like you want to run the Tez example "OrderedWordCount" using the tez-examples*.jar file. The OrderedWordCount example is part of the Tez examples and demonstrates how to perform a word count with ordering.

Assuming you have Tez installed on your system, you can follow these steps:

export TEZ_CONF_DIR=/etc/tez/conf/
export TEZ_HOME=/opt/cloudera/parcels/CDH/lib/tez/
export HADOOP_CLASSPATH=${TEZ_CONF_DIR}:${TEZ_HOME}/bin/*:${TEZ_HOME}/*

yarn jar ${TEZ_HOME}/bin/tez-examples-*.jar orderedwordcount /somewhere/input /somewhere/output

Cloudera Community

Support Questions

tez not working properly