WARNING: Use "yarn jar" to launch YARN applications. 16/10/07 03:40:52 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 16/10/07 03:40:52 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 16/10/07 03:40:52 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 2016-10-07 03:40:52,099 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.4.0.0-169 (rexported) compiled Feb 10 2016, 07:50:04 2016-10-07 03:40:52,099 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/pig_1475811652097.log 2016-10-07 03:40:53,107 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found 2016-10-07 03:40:53,345 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020 2016-10-07 03:40:54,460 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-7c309476-cd28-40ef-a21e-5075d41d00e0 2016-10-07 03:40:54,994 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:40:55,252 [main] INFO org.apache.pig.backend.hadoop.ATSService - Created ATS Hook 2016-10-07 03:40:56,559 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 2016-10-07 03:40:56,620 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 2016-10-07 03:40:56,689 [main] INFO hive.metastore - Connected to metastore. 2016-10-07 03:40:57,222 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 2016-10-07 03:40:57,226 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 2016-10-07 03:40:57,227 [main] INFO hive.metastore - Connected to metastore. 2016-10-07 03:40:57,304 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2016-10-07 03:40:57,359 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 2016-10-07 03:40:57,391 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 2016-10-07 03:40:57,508 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false 2016-10-07 03:40:57,533 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1 2016-10-07 03:40:57,533 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 2016-10-07 03:40:57,735 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:40:57,755 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:40:58,019 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRScriptState - Pig script settings are added to the job 2016-10-07 03:40:58,023 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 2016-10-07 03:40:58,095 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 2016-10-07 03:40:58,182 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 2016-10-07 03:40:58,184 [main] INFO hive.metastore - Connected to metastore. 2016-10-07 03:40:58,298 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - This job cannot be converted run in-process 2016-10-07 03:40:59,040 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/hive-metastore-1.2.1000.2.4.0.0-169.jar to DistributedCache through /tmp/temp733219628/tmp-1132897697/hive-metastore-1.2.1000.2.4.0.0-169.jar 2016-10-07 03:40:59,085 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/libthrift-0.9.2.jar to DistributedCache through /tmp/temp733219628/tmp1631817940/libthrift-0.9.2.jar 2016-10-07 03:40:59,232 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/hive-exec-1.2.1000.2.4.0.0-169.jar to DistributedCache through /tmp/temp733219628/tmp1567560112/hive-exec-1.2.1000.2.4.0.0-169.jar 2016-10-07 03:40:59,254 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/libfb303-0.9.2.jar to DistributedCache through /tmp/temp733219628/tmp71643758/libfb303-0.9.2.jar 2016-10-07 03:40:59,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/jdo-api-3.0.1.jar to DistributedCache through /tmp/temp733219628/tmp-1036096039/jdo-api-3.0.1.jar 2016-10-07 03:40:59,295 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/lib/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar to DistributedCache through /tmp/temp733219628/tmp794408512/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar 2016-10-07 03:40:59,316 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1000.2.4.0.0-169.jar to DistributedCache through /tmp/temp733219628/tmp436538215/hive-hcatalog-core-1.2.1000.2.4.0.0-169.jar 2016-10-07 03:40:59,342 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/usercache/user3/appcache/application_1475806680839_0016/container_e42_1475806680839_0016_01_000002/hive.tar.gz/hive/hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1000.2.4.0.0-169.jar to DistributedCache through /tmp/temp733219628/tmp230177227/hive-hcatalog-pig-adapter-1.2.1000.2.4.0.0-169.jar 2016-10-07 03:40:59,834 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/filecache/11/pig.tar.gz/pig/pig-0.15.0.2.4.0.0-169-core-h2.jar to DistributedCache through /tmp/temp733219628/tmp17275167/pig-0.15.0.2.4.0.0-169-core-h2.jar 2016-10-07 03:40:59,856 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/filecache/11/pig.tar.gz/pig/lib/automaton-1.11-8.jar to DistributedCache through /tmp/temp733219628/tmp-1953100604/automaton-1.11-8.jar 2016-10-07 03:40:59,875 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/hadoop/yarn/local/filecache/11/pig.tar.gz/pig/lib/antlr-runtime-3.4.jar to DistributedCache through /tmp/temp733219628/tmp-613360465/antlr-runtime-3.4.jar 2016-10-07 03:40:59,918 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job 2016-10-07 03:40:59,982 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. 2016-10-07 03:41:00,138 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:00,139 [JobControl] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:00,246 [JobControl] WARN org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 2016-10-07 03:41:00,367 [JobControl] INFO org.apache.hadoop.hive.ql.log.PerfLogger - 2016-10-07 03:41:00,372 [JobControl] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - ORC pushdown predicate: null 2016-10-07 03:41:00,387 [JobControl] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - FooterCacheHitRatio: 0/0 2016-10-07 03:41:00,387 [JobControl] INFO org.apache.hadoop.hive.ql.log.PerfLogger - 2016-10-07 03:41:00,388 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 0 2016-10-07 03:41:00,434 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - number of splits:0 2016-10-07 03:41:00,559 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1475806680839_0017 2016-10-07 03:41:00,559 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Kind: mapreduce.job, Service: job_1475806680839_0016, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@21e056e9) 2016-10-07 03:41:00,737 [JobControl] INFO org.apache.hadoop.mapred.YARNRunner - Job jar is not present. Not adding any jar to the list of resources. 2016-10-07 03:41:00,940 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1475806680839_0017 2016-10-07 03:41:00,996 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1475806680839_0017/ 2016-10-07 03:41:00,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1475806680839_0017 2016-10-07 03:41:00,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases A 2016-10-07 03:41:00,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4] C: R: 2016-10-07 03:41:01,018 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 2016-10-07 03:41:01,018 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1475806680839_0017] 2016-10-07 03:41:10,041 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:10,141 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1475806680839_0017] 2016-10-07 03:41:11,271 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:11,271 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:11,282 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:11,435 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:11,435 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:11,443 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:11,675 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:11,676 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:11,686 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:11,889 [main] WARN mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2016-10-07 03:41:11,930 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 2016-10-07 03:41:11,933 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: HadoopVersion PigVersion UserId StartedAt FinishedAt Features 2.7.1.2.4.0.0-169 0.15.0.2.4.0.0-169 yarn 2016-10-07 03:40:58 2016-10-07 03:41:11 UNKNOWN Success! Job Stats (time in seconds): JobId Maps Reduces MaxMapTime MinMapTime AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs job_1475806680839_0017 0 0 0 0 0 0 0 0 0 0 A MAP_ONLY hdfs://sandbox.hortonworks.com:8020/tmp/temp733219628/tmp1293276776, Input(s): Successfully read 0 records from: "orctable1" Output(s): Successfully stored 0 records in: "hdfs://sandbox.hortonworks.com:8020/tmp/temp733219628/tmp1293276776" Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager spill count : 0 Total bags proactively spilled: 0 Total records proactively spilled: 0 Job DAG: job_1475806680839_0017 2016-10-07 03:41:12,090 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:12,090 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:12,098 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:12,240 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:12,240 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:12,248 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:12,393 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 2016-10-07 03:41:12,393 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.0.126:8050 2016-10-07 03:41:12,399 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2016-10-07 03:41:12,451 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success! 2016-10-07 03:41:12,455 [main] WARN org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized 2016-10-07 03:41:12,465 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 0 2016-10-07 03:41:12,465 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 0 2016-10-07 03:41:12,510 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 2016-10-07 03:41:12,512 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083 2016-10-07 03:41:12,514 [main] INFO hive.metastore - Connected to metastore. 2016-10-07 03:41:12,579 [main] INFO org.apache.pig.Main - Pig script completed in 20 seconds and 591 milliseconds (20591 ms)