Created 07-28-2017 04:17 AM
Hi, im trying to execute pig script using tez in pig view it fails, but when i run in command line it works fine
ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:166) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2017-07-28 12:02:44,415 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:166) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more
Created 07-28-2017 04:31 AM
You are getting the error showing the following message:
TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined
So can you please check and share the "tez.lib.uris" property to the above file location under Ambari > Hive configs > (custom hive-site) is it pointing to correct tar file? Example:
tez.lib.uris = /hdp/apps/${hdp.version}/tez/tez.tar.gz
- Also please check if the path for the "tez-site.xml" is added in the "webhcat-site.xml" in Hive configs as following or not:
templeton.libjars=/usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar/,/etc/tez/conf/tez-site.xml
.
Created 07-28-2017 04:31 AM
You are getting the error showing the following message:
TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined
So can you please check and share the "tez.lib.uris" property to the above file location under Ambari > Hive configs > (custom hive-site) is it pointing to correct tar file? Example:
tez.lib.uris = /hdp/apps/${hdp.version}/tez/tez.tar.gz
- Also please check if the path for the "tez-site.xml" is added in the "webhcat-site.xml" in Hive configs as following or not:
templeton.libjars=/usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar/,/etc/tez/conf/tez-site.xml
.
Created 07-28-2017 09:02 AM
thank you it works for me.. but can u explain why we have to add this in the configuration?