Hi, We have configured a cluster with the services of HDFS and Yarn+MapReduce2 along with the required services using HDP 2.6. When we tried to install other services like Hive, Tez, Hive, and Spark, the installation was successful but when the starting of the services was hindered with the following warning message. Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py", line 141, in <module> PigServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py", line 85, in service_check logoutput = True File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:30:25,626 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:30:25,627 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399425624.log 2018-04-10 17:30:26,356 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:30:26,467 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:30:27,022 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-9a3761a2-f84c-431a-bce4-0c9e9506ea69 2018-04-10 17:30:27,475 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:30:27,578 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:28,506 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:28,540 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:28,568 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:28,615 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:28,771 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp1963801132 2018-04-10 17:30:28,827 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:28,968 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:28,973 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:28,973 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:29,016 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:30,965 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:31,068 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:31,245 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:31,299 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:31,406 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:31,410 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:31,413 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:31,415 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:31,417 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:32,003 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:32,012 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:28 FinishedAt: 2018-04-10 17:30:32 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:32,017 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399425624.log 2018-04-10 17:30:32,050 [main] INFO org.apache.pig.Main - Pig script completed in 6 seconds and 752 milliseconds (6752 ms) 2018-04-10 17:30:32,050 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread poolstdout: /var/lib/ambari-agent/data/output-92.txt 2018-04-10 17:29:51,617 - Using hadoop conf dir: /usr/hdp/2.6.4.0-91/hadoop/conf 2018-04-10 17:29:51,635 - HdfsResource['/user/ambari-qa'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp'], 'mode': 0770} 2018-04-10 17:29:51,643 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpEe53mC 2>/tmp/tmpU1T7wf''] {'logoutput': None, 'quiet': False} 2018-04-10 17:29:51,771 - call returned (0, '') 2018-04-10 17:29:51,773 - HdfsResource['/user/ambari-qa/pigsmoke.out'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'directory', 'action': ['delete_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']} 2018-04-10 17:29:51,775 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/pigsmoke.out?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmp4KY1oF 2>/tmp/tmpIj7sIe''] {'logoutput': None, 'quiet': False} 2018-04-10 17:29:51,878 - call returned (0, '') 2018-04-10 17:29:51,880 - HdfsResource['/user/ambari-qa/passwd'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'source': '/etc/passwd', 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']} 2018-04-10 17:29:51,882 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpcIyi4H 2>/tmp/tmpGJ0R33''] {'logoutput': None, 'quiet': False} 2018-04-10 17:29:51,969 - call returned (0, '') 2018-04-10 17:29:51,970 - Creating new file /user/ambari-qa/passwd in DFS 2018-04-10 17:29:51,971 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X PUT --data-binary @/etc/passwd -H '"'"'Content-Type: application/octet-stream'"'"' '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=CREATE&user.name=hdfs&overwrite=True'"'"' 1>/tmp/tmpxtNcG3 2>/tmp/tmpe2r7xN''] {'logoutput': None, 'quiet': False} 2018-04-10 17:29:52,325 - call returned (0, '') 2018-04-10 17:29:52,328 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X PUT '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=SETOWNER&owner=ambari-qa&group=&user.name=hdfs'"'"' 1>/tmp/tmp0ownP5 2>/tmp/tmp8rxp5Z''] {'logoutput': None, 'quiet': False} 2018-04-10 17:29:52,507 - call returned (0, '') 2018-04-10 17:29:52,508 - HdfsResource[None] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']} 2018-04-10 17:29:52,509 - File['/var/lib/ambari-agent/tmp/pigSmoke.sh'] {'content': StaticFile('pigSmoke.sh'), 'mode': 0755} 2018-04-10 17:29:52,512 - Writing File['/var/lib/ambari-agent/tmp/pigSmoke.sh'] because it doesn't exist 2018-04-10 17:29:52,512 - Changing permission for /var/lib/ambari-agent/tmp/pigSmoke.sh from 644 to 755 2018-04-10 17:29:52,513 - Execute['pig /var/lib/ambari-agent/tmp/pigSmoke.sh'] {'logoutput': True, 'path': [u'/usr/hdp/current/pig-client/bin:/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5} 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399395857.log 2018-04-10 17:29:57,163 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:29:57,319 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:29:58,409 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-3f749dfe-3f31-4f30-a128-bbe57cabd2b5 2018-04-10 17:29:59,179 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:29:59,387 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:00,294 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:00,329 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:00,361 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:00,431 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:00,634 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp586283164 2018-04-10 17:30:00,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:00,889 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:00,896 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:00,897 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:00,940 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:02,820 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:02,903 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:03,028 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:03,066 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:03,191 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:03,195 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:03,198 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:03,200 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:03,203 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:03,850 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:03,859 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:00 FinishedAt: 2018-04-10 17:30:03 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:03,863 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399395857.log 2018-04-10 17:30:03,896 [main] INFO org.apache.pig.Main - Pig script completed in 8 seconds and 385 milliseconds (8385 ms) 2018-04-10 17:30:03,896 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool 2018-04-10 17:30:04,648 - Retrying after 5 seconds. Reason: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399395857.log 2018-04-10 17:29:57,163 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:29:57,319 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:29:58,409 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-3f749dfe-3f31-4f30-a128-bbe57cabd2b5 2018-04-10 17:29:59,179 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:29:59,387 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:00,294 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:00,329 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:00,361 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:00,431 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:00,634 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp586283164 2018-04-10 17:30:00,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:00,889 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:00,896 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:00,897 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:00,940 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:02,820 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:02,903 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:03,028 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:03,066 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:03,191 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:03,195 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:03,198 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:03,200 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:03,203 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:03,850 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:03,859 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:00 FinishedAt: 2018-04-10 17:30:03 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:03,863 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399395857.log 2018-04-10 17:30:03,896 [main] INFO org.apache.pig.Main - Pig script completed in 8 seconds and 385 milliseconds (8385 ms) 2018-04-10 17:30:03,896 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399411280.log 2018-04-10 17:30:11,992 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:30:12,151 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:30:12,728 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-81418431-0bca-4f2a-be22-0d854740ba23 2018-04-10 17:30:13,064 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:30:13,172 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:13,959 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:13,991 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:14,021 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:14,071 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:14,225 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp2038674231 2018-04-10 17:30:14,255 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:14,383 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:14,388 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:14,388 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:14,431 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:17,109 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:17,196 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:17,317 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:17,367 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:17,492 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:17,496 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:17,500 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:17,502 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:17,504 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:18,141 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:18,150 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:14 FinishedAt: 2018-04-10 17:30:18 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:18,154 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399411280.log 2018-04-10 17:30:18,187 [main] INFO org.apache.pig.Main - Pig script completed in 7 seconds and 208 milliseconds (7208 ms) 2018-04-10 17:30:18,187 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool 2018-04-10 17:30:18,895 - Retrying after 5 seconds. Reason: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399411280.log 2018-04-10 17:30:11,992 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:30:12,151 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:30:12,728 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-81418431-0bca-4f2a-be22-0d854740ba23 2018-04-10 17:30:13,064 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:30:13,172 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:13,959 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:13,991 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:14,021 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:14,071 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:14,225 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp2038674231 2018-04-10 17:30:14,255 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:14,383 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:14,388 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:14,388 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:14,431 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:17,109 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:17,196 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:17,317 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:17,367 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:17,492 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:17,496 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:17,500 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:17,502 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:17,504 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:18,141 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:18,150 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:14 FinishedAt: 2018-04-10 17:30:18 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:18,154 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399411280.log 2018-04-10 17:30:18,187 [main] INFO org.apache.pig.Main - Pig script completed in 7 seconds and 208 milliseconds (7208 ms) 2018-04-10 17:30:18,187 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 2018-04-10 17:30:25,626 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08 2018-04-10 17:30:25,627 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399425624.log 2018-04-10 17:30:26,356 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found 2018-04-10 17:30:26,467 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020 2018-04-10 17:30:27,022 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-9a3761a2-f84c-431a-bce4-0c9e9506ea69 2018-04-10 17:30:27,475 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/ 2018-04-10 17:30:27,578 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 2018-04-10 17:30:28,506 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2018-04-10 17:30:28,540 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2018-04-10 17:30:28,568 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2018-04-10 17:30:28,615 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128 2018-04-10 17:30:28,771 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp1963801132 2018-04-10 17:30:28,827 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 2018-04-10 17:30:28,968 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 2018-04-10 17:30:28,973 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2018-04-10 17:30:28,973 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 2018-04-10 17:30:29,016 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4] 2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: 2018-04-10 17:30:30,965 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 2018-04-10 17:30:31,068 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 2018-04-10 17:30:31,245 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ] 2018-04-10 17:30:31,299 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050 2018-04-10 17:30:31,406 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200 2018-04-10 17:30:31,410 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session. 2018-04-10 17:30:31,413 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz 2018-04-10 17:30:31,415 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-04-10 17:30:31,417 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446) at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454) at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770) at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831) at org.apache.tez.client.TezClient.start(TezClient.java:355) at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102) ... 8 more 2018-04-10 17:30:32,003 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2018-04-10 17:30:32,012 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.7.3.2.6.4.0-91 PigVersion: 0.16.0.2.6.4.0-91 TezVersion: 0.7.0.2.6.4.0-91 UserId: ambari-qa FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh StartedAt: 2018-04-10 17:30:28 FinishedAt: 2018-04-10 17:30:32 Features: UNKNOWN Failed! DAG 0: Name: PigLatin:pigSmoke.sh-0_scope-0 ApplicationId: null TotalLaunchedTasks: -1 FileBytesRead: -1 FileBytesWritten: -1 HdfsBytesRead: 0 HdfsBytesWritten: 0 SpillableMemoryManager spill count: 0 Bags proactively spilled: 0 Records proactively spilled: 0 DAG Plan: Tez vertex scope-5 Input(s): Output(s): 2018-04-10 17:30:32,017 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message Details at logfile: /home/ambari-qa/pig_1523399425624.log 2018-04-10 17:30:32,050 [main] INFO org.apache.pig.Main - Pig script completed in 6 seconds and 752 milliseconds (6752 ms) 2018-04-10 17:30:32,050 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool Command failed after 1 tries Could someone please look into it and advise for the next steps. Thanks. ... View more