ls: cannot access /hadoop/yarn/local/usercache/subrata/appcache/application_1489855879213_0001/container_e24_1489855879213_0001_01_000002/hive.tar.gz/hive/lib/slf4j-api-*.jar: No such file or directory
ls: cannot access /hadoop/yarn/local/usercache/subrata/appcache/application_1489855879213_0001/container_e24_1489855879213_0001_01_000002/hive.tar.gz/hive/hcatalog/lib/*hbase-storage-handler-*.jar: No such file or directory
WARNING: Use "yarn jar" to launch YARN applications.
17/03/18 22:43:16 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
17/03/18 22:43:16 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
17/03/18 22:43:16 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
17/03/18 22:43:16 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
17/03/18 22:43:16 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2017-03-18 22:43:16,323 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0.2.5.3.0-37 (rexported) compiled Nov 30 2016, 03:39:58
2017-03-18 22:43:16,323 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/subrata/appcache/application_1489855879213_0001/container_e24_1489855879213_0001_01_000002/pig_1489857196322.log
2017-03-18 22:43:16,883 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2017-03-18 22:43:17,002 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://pc-1.thenet.edu:8020
2017-03-18 22:43:17,713 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-684fa29b-1138-49f2-a6be-3b4ebd7babd5
2017-03-18 22:43:18,097 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://pc-1.thenet.edu:8188/ws/v1/timeline/
2017-03-18 22:43:18,201 [main] INFO org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook
2017-03-18 22:43:19,004 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:19,056 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:19,090 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:20,386 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:20,390 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:20,394 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:20,731 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_FLOAT 1 time(s).
2017-03-18 22:43:20,788 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:20,822 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:20,823 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:21,120 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:21,149 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:21,150 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:21,270 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: HASH_JOIN,GROUP_BY,FILTER
2017-03-18 22:43:21,293 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2017-03-18 22:43:21,318 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2017-03-18 22:43:21,350 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 716177408 to monitor. collectionUsageThreshold = 501324192, usageThreshold = 501324192
2017-03-18 22:43:21,393 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for a: $0, $3, $4, $5, $6, $7, $8, $9
2017-03-18 22:43:21,569 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/yarn/staging and resources directory is /tmp/temp-816712545
2017-03-18 22:43:21,641 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2017-03-18 22:43:21,712 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
2017-03-18 22:43:21,830 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:21,836 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:21,837 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:22,114 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2017-03-18 22:43:22,123 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - ORC pushdown predicate: null
2017-03-18 22:43:22,146 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcUtils - Using schema evolution configuration variables schema.evolution.columns [truckid, driverid, event, latitude, longitude, city, state, velocity, event_ind, idling_ind] / schema.evolution.columns.types [string, string, string, string, string, string, string, string, string, string] (isAcid false)
2017-03-18 22:43:22,492 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - FooterCacheHitRatio: 0/2
2017-03-18 22:43:22,492 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2017-03-18 22:43:22,494 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2017-03-18 22:43:22,630 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:22,644 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:22,653 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:22,767 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2017-03-18 22:43:22,767 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - ORC pushdown predicate: null
2017-03-18 22:43:22,767 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcUtils - Using schema evolution configuration variables schema.evolution.columns [driverid, totmiles] / schema.evolution.columns.types [string, bigint] (isAcid false)
2017-03-18 22:43:22,788 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - FooterCacheHitRatio: 0/2
2017-03-18 22:43:22,788 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2017-03-18 22:43:22,789 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libfb303-0.9.3.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.5.3.0-37-core-h2.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libthrift-0.9.3.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: jdo-api-3.0.1.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-pig-adapter-1.2.1000.2.5.3.0-37.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hbase-handler-1.2.1000.2.5.3.0-37.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-exec-1.2.1000.2.5.3.0-37.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-core-1.2.1000.2.5.3.0-37.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2017-03-18 22:43:24,523 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-metastore-1.2.1000.2.5.3.0-37.jar
2017-03-18 22:43:24,668 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-51: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2017-03-18 22:43:24,669 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: a,b,c,d,e
2017-03-18 22:43:24,669 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: a[1,4],b[2,4],c[3,4],e[5,4],d[4,4]
2017-03-18 22:43:24,669 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2017-03-18 22:43:24,727 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Set auto parallelism for vertex scope-52
2017-03-18 22:43:24,728 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-52: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2017-03-18 22:43:24,728 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: e,h
2017-03-18 22:43:24,728 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: e[5,4],h[7,4]
2017-03-18 22:43:24,728 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: GROUP_BY
2017-03-18 22:43:24,784 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-53: parallelism=1, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2017-03-18 22:43:24,784 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: g,h
2017-03-18 22:43:24,784 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: g[6,4],h[7,4]
2017-03-18 22:43:24,784 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:
2017-03-18 22:43:24,845 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Set auto parallelism for vertex scope-54
2017-03-18 22:43:24,846 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-54: parallelism=2, memory=1024, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
2017-03-18 22:43:24,846 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: final_data,h
2017-03-18 22:43:24,846 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: h[7,4],final_data[8,13]
2017-03-18 22:43:24,846 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex: HASH_JOIN
2017-03-18 22:43:24,918 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2017-03-18 22:43:24,923 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://pc-1.thenet.edu:9083
2017-03-18 22:43:24,924 [main] INFO hive.metastore - Connected to metastore.
2017-03-18 22:43:24,949 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 5
2017-03-18 22:43:25,010 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2017-03-18 22:43:25,104 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.5.3.0-37, revision=2269be0a12263567ecfea0705e3c72244baa5830, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2016-11-29T18:40:01Z ]
2017-03-18 22:43:25,259 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://pc-1.thenet.edu:8188/ws/v1/timeline/
2017-03-18 22:43:25,266 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at pc-1.thenet.edu/127.0.1.1:8050
2017-03-18 22:43:25,445 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at pc-1.thenet.edu/127.0.1.1:10200
2017-03-18 22:43:25,458 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Using org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager to manage Timeline ACLs
2017-03-18 22:43:25,631 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://pc-1.thenet.edu:8188/ws/v1/timeline/
2017-03-18 22:43:25,640 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2017-03-18 22:43:25,648 [PigTezLauncher-0] INFO org.apache.tez.common.security.TokenCache - Merging additional tokens from binary file, binaryFileName=/hadoop/yarn/local/usercache/subrata/appcache/application_1489855879213_0001/container_e24_1489855879213_0001_01_000002/container_tokens
2017-03-18 22:43:25,649 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.5.3.0-37/tez/tez.tar.gz
2017-03-18 22:43:25,828 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez system stage directory hdfs://pc-1.thenet.edu:8020/tmp/yarn/staging/.tez/application_1489855879213_0002 doesn't exist and is created
2017-03-18 22:43:26,569 [PigTezLauncher-0] INFO org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager - Created Timeline Domain for History ACLs, domainId=Tez_ATS_application_1489855879213_0002
2017-03-18 22:43:27,002 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1489855879213_0002
2017-03-18 22:43:27,005 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - The url to track the Tez Session: http://pc-1.thenet.edu:8088/proxy/application_1489855879213_0002/
------------------------------------------------------------------------------------------------------------
Following portion of the log is after I hit the 'KILL' button -->>>>
------------------------------------------------------------------------------------------------------------
2017-03-18 23:31:13,301 [Thread-3] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
2017-03-18 23:31:13,302 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Exception while waiting for Tez client to be ready
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:785)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:105)
at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:232)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(Fu