Member since
03-20-2018
2
Posts
0
Kudos Received
0
Solutions
05-15-2018
08:54 PM
Hi, We have configured a cluster with the services of HDFS and Yarn+MapReduce2 along with the required services using HDP 2.6. When we tried to install other services like Hive, Tez, Hive, and Spark, the installation was successful but when the starting of the services was hindered with the following warning message. Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py", line 141, in <module>
PigServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py", line 85, in service_check
logoutput = True
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:30:25,626 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:30:25,627 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399425624.log
2018-04-10 17:30:26,356 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:30:26,467 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:30:27,022 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-9a3761a2-f84c-431a-bce4-0c9e9506ea69
2018-04-10 17:30:27,475 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:30:27,578 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:28,506 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:28,540 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:28,568 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:28,615 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:28,771 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp1963801132
2018-04-10 17:30:28,827 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:28,968 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:28,973 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:28,973 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:29,016 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:30,965 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:31,068 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:31,245 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:31,299 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:31,406 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:31,410 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:31,413 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:31,415 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:31,417 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:32,003 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:32,012 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:28
FinishedAt: 2018-04-10 17:30:32
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:32,017 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399425624.log
2018-04-10 17:30:32,050 [main] INFO org.apache.pig.Main - Pig script completed in 6 seconds and 752 milliseconds (6752 ms)
2018-04-10 17:30:32,050 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread poolstdout: /var/lib/ambari-agent/data/output-92.txt
2018-04-10 17:29:51,617 - Using hadoop conf dir: /usr/hdp/2.6.4.0-91/hadoop/conf
2018-04-10 17:29:51,635 - HdfsResource['/user/ambari-qa'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp'], 'mode': 0770}
2018-04-10 17:29:51,643 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpEe53mC 2>/tmp/tmpU1T7wf''] {'logoutput': None, 'quiet': False}
2018-04-10 17:29:51,771 - call returned (0, '')
2018-04-10 17:29:51,773 - HdfsResource['/user/ambari-qa/pigsmoke.out'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'directory', 'action': ['delete_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']}
2018-04-10 17:29:51,775 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/pigsmoke.out?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmp4KY1oF 2>/tmp/tmpIj7sIe''] {'logoutput': None, 'quiet': False}
2018-04-10 17:29:51,878 - call returned (0, '')
2018-04-10 17:29:51,880 - HdfsResource['/user/ambari-qa/passwd'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'source': '/etc/passwd', 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']}
2018-04-10 17:29:51,882 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpcIyi4H 2>/tmp/tmpGJ0R33''] {'logoutput': None, 'quiet': False}
2018-04-10 17:29:51,969 - call returned (0, '')
2018-04-10 17:29:51,970 - Creating new file /user/ambari-qa/passwd in DFS
2018-04-10 17:29:51,971 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X PUT --data-binary @/etc/passwd -H '"'"'Content-Type: application/octet-stream'"'"' '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=CREATE&user.name=hdfs&overwrite=True'"'"' 1>/tmp/tmpxtNcG3 2>/tmp/tmpe2r7xN''] {'logoutput': None, 'quiet': False}
2018-04-10 17:29:52,325 - call returned (0, '')
2018-04-10 17:29:52,328 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X PUT '"'"'http://usmdhdplt001.vistex.com:50070/webhdfs/v1/user/ambari-qa/passwd?op=SETOWNER&owner=ambari-qa&group=&user.name=hdfs'"'"' 1>/tmp/tmp0ownP5 2>/tmp/tmp8rxp5Z''] {'logoutput': None, 'quiet': False}
2018-04-10 17:29:52,507 - call returned (0, '')
2018-04-10 17:29:52,508 - HdfsResource[None] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/2.6.4.0-91/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': '', 'default_fs': 'hdfs://usmdhdplt001.vistex.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir': '/usr/hdp/2.6.4.0-91/hadoop/conf', 'immutable_paths': [u'/apps/hive/warehouse', u'/mr-history/done', u'/app-logs', u'/tmp']}
2018-04-10 17:29:52,509 - File['/var/lib/ambari-agent/tmp/pigSmoke.sh'] {'content': StaticFile('pigSmoke.sh'), 'mode': 0755}
2018-04-10 17:29:52,512 - Writing File['/var/lib/ambari-agent/tmp/pigSmoke.sh'] because it doesn't exist
2018-04-10 17:29:52,512 - Changing permission for /var/lib/ambari-agent/tmp/pigSmoke.sh from 644 to 755
2018-04-10 17:29:52,513 - Execute['pig /var/lib/ambari-agent/tmp/pigSmoke.sh'] {'logoutput': True, 'path': [u'/usr/hdp/current/pig-client/bin:/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5}
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399395857.log
2018-04-10 17:29:57,163 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:29:57,319 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:29:58,409 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-3f749dfe-3f31-4f30-a128-bbe57cabd2b5
2018-04-10 17:29:59,179 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:29:59,387 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:00,294 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:00,329 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:00,361 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:00,431 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:00,634 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp586283164
2018-04-10 17:30:00,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:00,889 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:00,896 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:00,897 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:00,940 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:02,820 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:02,903 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:03,028 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:03,066 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:03,191 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:03,195 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:03,198 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:03,200 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:03,203 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:03,850 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:03,859 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:00
FinishedAt: 2018-04-10 17:30:03
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:03,863 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399395857.log
2018-04-10 17:30:03,896 [main] INFO org.apache.pig.Main - Pig script completed in 8 seconds and 385 milliseconds (8385 ms)
2018-04-10 17:30:03,896 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
2018-04-10 17:30:04,648 - Retrying after 5 seconds. Reason: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:29:55 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:29:55,860 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399395857.log
2018-04-10 17:29:57,163 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:29:57,319 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:29:58,409 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-3f749dfe-3f31-4f30-a128-bbe57cabd2b5
2018-04-10 17:29:59,179 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:29:59,387 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:00,294 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:00,329 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:00,361 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:00,431 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:00,634 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp586283164
2018-04-10 17:30:00,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:00,889 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:00,896 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:00,897 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:00,940 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:02,364 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:02,739 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:02,820 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:02,903 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:03,028 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:03,066 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:03,191 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:03,195 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:03,198 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:03,200 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:03,203 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:03,850 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:03,859 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:00
FinishedAt: 2018-04-10 17:30:03
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:03,863 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399395857.log
2018-04-10 17:30:03,896 [main] INFO org.apache.pig.Main - Pig script completed in 8 seconds and 385 milliseconds (8385 ms)
2018-04-10 17:30:03,896 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399411280.log
2018-04-10 17:30:11,992 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:30:12,151 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:30:12,728 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-81418431-0bca-4f2a-be22-0d854740ba23
2018-04-10 17:30:13,064 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:30:13,172 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:13,959 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:13,991 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:14,021 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:14,071 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:14,225 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp2038674231
2018-04-10 17:30:14,255 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:14,383 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:14,388 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:14,388 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:14,431 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:17,109 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:17,196 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:17,317 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:17,367 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:17,492 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:17,496 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:17,500 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:17,502 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:17,504 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:18,141 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:18,150 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:14
FinishedAt: 2018-04-10 17:30:18
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:18,154 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399411280.log
2018-04-10 17:30:18,187 [main] INFO org.apache.pig.Main - Pig script completed in 7 seconds and 208 milliseconds (7208 ms)
2018-04-10 17:30:18,187 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
2018-04-10 17:30:18,895 - Retrying after 5 seconds. Reason: Execution of 'pig /var/lib/ambari-agent/tmp/pigSmoke.sh' returned 2. 18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:30:11 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:30:11,283 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399411280.log
2018-04-10 17:30:11,992 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:30:12,151 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:30:12,728 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-81418431-0bca-4f2a-be22-0d854740ba23
2018-04-10 17:30:13,064 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:30:13,172 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:13,959 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:13,991 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:14,021 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:14,071 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:14,225 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp2038674231
2018-04-10 17:30:14,255 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:14,383 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:14,388 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:14,388 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:14,431 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:16,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:17,027 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:17,028 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:17,109 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:17,196 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:17,317 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:17,367 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:17,492 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:17,496 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:17,500 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:17,502 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:17,504 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:18,141 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:18,150 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:14
FinishedAt: 2018-04-10 17:30:18
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:18,154 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399411280.log
2018-04-10 17:30:18,187 [main] INFO org.apache.pig.Main - Pig script completed in 7 seconds and 208 milliseconds (7208 ms)
2018-04-10 17:30:18,187 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
18/04/10 17:30:25 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2018-04-10 17:30:25,626 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.6.4.0-91 (rUnversioned directory) compiled Jan 04 2018, 10:33:08
2018-04-10 17:30:25,627 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ambari-qa/pig_1523399425624.log
2018-04-10 17:30:26,356 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ambari-qa/.pigbootup not found
2018-04-10 17:30:26,467 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://usmdhdplt001.vistex.com:8020
2018-04-10 17:30:27,022 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-pigSmoke.sh-9a3761a2-f84c-431a-bce4-0c9e9506ea69
2018-04-10 17:30:27,475 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://usmdhdplt001.vistex.com:8188/ws/v1/timeline/
2018-04-10 17:30:27,578 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2018-04-10 17:30:28,506 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2018-04-10 17:30:28,540 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2018-04-10 17:30:28,568 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2018-04-10 17:30:28,615 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 699400192 to monitor. collectionUsageThreshold = 489580128, usageThreshold = 489580128
2018-04-10 17:30:28,771 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/ambari-qa/staging and resources directory is /tmp/temp1963801132
2018-04-10 17:30:28,827 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2018-04-10 17:30:28,968 [main] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2018-04-10 17:30:28,973 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2018-04-10 17:30:28,973 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2018-04-10 17:30:29,016 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: joda-time-2.9.4.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.6.4.0-91-core-h2.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2018-04-10 17:30:30,463 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-5: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A,B
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[16,4],B[17,4]
2018-04-10 17:30:30,839 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2018-04-10 17:30:30,965 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1
2018-04-10 17:30:31,068 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2018-04-10 17:30:31,245 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.6.4.0-91, revision=0daef967e122a98f83b824f3b14991e792f5dd4d, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T09:08:42Z ]
2018-04-10 17:30:31,299 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at usmdhdplt001.vistex.com/10.1.180.169:8050
2018-04-10 17:30:31,406 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at usmdhdplt001.vistex.com/10.1.180.169:10200
2018-04-10 17:30:31,410 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2018-04-10 17:30:31,413 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
2018-04-10 17:30:31,415 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-04-10 17:30:31,417 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:111)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:234)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: /hdp/apps/2.6.4.0-91/tez/tez.tar.gz
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1446)
at org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:770)
at org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:132)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
at org.apache.tez.client.TezClient.start(TezClient.java:355)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:102)
... 8 more
2018-04-10 17:30:32,003 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2018-04-10 17:30:32,012 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.3.2.6.4.0-91
PigVersion: 0.16.0.2.6.4.0-91
TezVersion: 0.7.0.2.6.4.0-91
UserId: ambari-qa
FileName: /var/lib/ambari-agent/tmp/pigSmoke.sh
StartedAt: 2018-04-10 17:30:28
FinishedAt: 2018-04-10 17:30:32
Features: UNKNOWN
Failed!
DAG 0:
Name: PigLatin:pigSmoke.sh-0_scope-0
ApplicationId: null
TotalLaunchedTasks: -1
FileBytesRead: -1
FileBytesWritten: -1
HdfsBytesRead: 0
HdfsBytesWritten: 0
SpillableMemoryManager spill count: 0
Bags proactively spilled: 0
Records proactively spilled: 0
DAG Plan:
Tez vertex scope-5
Input(s):
Output(s):
2018-04-10 17:30:32,017 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/ambari-qa/pig_1523399425624.log
2018-04-10 17:30:32,050 [main] INFO org.apache.pig.Main - Pig script completed in 6 seconds and 752 milliseconds (6752 ms)
2018-04-10 17:30:32,050 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
Command failed after 1 tries Could someone please look into it and advise for the next steps. Thanks.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
-
Apache Tez
03-20-2018
04:31 PM
Hi, I am a newbie to installing and working on HDP 2.6. We have setup a Linux server (OpenSuse 12.2) and are trying to install a cluster for testing purposes. During the cluster installation using ambari, there is an error that seems to prevent us from proceeding further. (PFA the image) I have looked around but do not seem to find a solution for this error. Could anyone help me out with this. Thanks.
... View more
Labels: