WARNING: Use "yarn jar" to launch YARN applications.
16/07/19 19:11:51 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/07/19 19:11:51 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/07/19 19:11:51 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL
16/07/19 19:11:51 INFO pig.ExecTypeProvider: Trying ExecType : TEZ
16/07/19 19:11:51 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
2016-07-19 19:11:52,092 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.4.0.0-169 (rexported) compiled Feb 10 2016, 07:50:04
2016-07-19 19:11:52,093 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/pig_1468955512051.log
2016-07-19 19:11:57,824 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-07-19 19:11:58,158 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-07-19 19:12:02,498 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-3910d5d6-b7d6-4739-b8d6-a42d82bd50d8
2016-07-19 19:12:04,474 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-07-19 19:12:06,105 [main] INFO org.apache.pig.backend.hadoop.ATSService - Created ATS Hook
2016-07-19 19:12:08,930 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:09,112 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:09,222 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:09,283 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:22,305 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:22,309 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:22,314 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:22,459 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:26,584 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_FLOAT 1 time(s).
2016-07-19 19:12:26,867 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:26,934 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:26,935 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:26,986 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:28,887 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:28,928 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:28,931 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:28,957 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:32,609 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: HASH_JOIN,GROUP_BY,FILTER
2016-07-19 19:12:32,737 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2016-07-19 19:12:32,926 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-07-19 19:12:33,123 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for a: $0, $3, $4, $5, $6, $7, $8, $9
2016-07-19 19:12:33,418 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tmp/yarn/staging and resources directory is /tmp/temp1542924427
2016-07-19 19:12:33,574 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false
2016-07-19 19:12:33,636 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
2016-07-19 19:12:33,840 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:33,860 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:33,862 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:33,903 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:35,058 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2016-07-19 19:12:35,095 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - ORC pushdown predicate: null
2016-07-19 19:12:35,413 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - FooterCacheHitRatio: 0/0
2016-07-19 19:12:35,414 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2016-07-19 19:12:35,422 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-07-19 19:12:35,513 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:35,535 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:35,542 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:35,544 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:35,887 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2016-07-19 19:12:35,888 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - ORC pushdown predicate: null
2016-07-19 19:12:35,895 [main] INFO org.apache.hadoop.hive.ql.io.orc.OrcInputFormat - FooterCacheHitRatio: 0/0
2016-07-19 19:12:35,895 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger -
2016-07-19 19:12:35,895 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-07-19 19:12:47,101 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-exec-1.2.1000.2.4.0.0-169.jar
2016-07-19 19:12:47,102 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libfb303-0.9.2.jar
2016-07-19 19:12:47,111 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libthrift-0.9.2.jar
2016-07-19 19:12:47,111 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: jdo-api-3.0.1.jar
2016-07-19 19:12:47,112 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-metastore-1.2.1000.2.4.0.0-169.jar
2016-07-19 19:12:47,112 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar
2016-07-19 19:12:47,112 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hbase-handler-1.2.1000.2.4.0.0-169.jar
2016-07-19 19:12:47,113 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-pig-adapter-1.2.1000.2.4.0.0-169.jar
2016-07-19 19:12:47,113 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.15.0.2.4.0.0-169-core-h2.jar
2016-07-19 19:12:47,113 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar
2016-07-19 19:12:47,113 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-core-1.2.1000.2.4.0.0-169.jar
2016-07-19 19:12:47,735 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:47,735 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:47,735 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:47,735 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:47,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:47,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:47,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:47,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:47,736 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:47,737 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:47,738 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:47,738 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:48,026 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - For vertex - scope-51: parallelism=1, memory=250, java opts=-Xmx256m
2016-07-19 19:12:48,274 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:48,274 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:48,275 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:48,276 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:48,276 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:48,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:48,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:48,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:48,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:48,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:48,283 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:48,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Set auto parallelism for vertex scope-52
2016-07-19 19:12:48,511 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - For vertex - scope-52: parallelism=1, memory=250, java opts=-Xmx256m
2016-07-19 19:12:48,627 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:48,627 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:48,627 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:48,627 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:48,627 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:48,628 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:48,629 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:48,629 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:48,629 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:48,629 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:48,772 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:48,772 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:48,772 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:48,772 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:48,772 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:48,773 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:48,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:48,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:48,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:48,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:48,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:48,822 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - For vertex - scope-53: parallelism=1, memory=250, java opts=-Xmx256m
2016-07-19 19:12:48,960 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:48,960 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:48,960 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:48,960 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:48,961 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:48,962 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:49,049 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Set auto parallelism for vertex scope-54
2016-07-19 19:12:49,050 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - For vertex - scope-54: parallelism=2, memory=250, java opts=-Xmx256m
2016-07-19 19:12:49,258 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-07-19 19:12:49,277 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-07-19 19:12:49,318 [main] WARN org.apache.hadoop.security.UserGroupInformation - No groups available for user maria_dev
2016-07-19 19:12:49,329 [main] INFO hive.metastore - Connected to metastore.
2016-07-19 19:12:49,881 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:49,881 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:49,881 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:49,882 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:49,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:49,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:49,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:49,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:49,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:49,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:49,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:49,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:49,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR setting mapreduce.reduce.shuffle.merge.percent
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.7 from MR setting mapreduce.reduce.shuffle.input.buffer.percent
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.mb to 64 from MR setting mapreduce.task.io.sort.mb
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 from MR setting mapreduce.reduce.shuffle.memory.limit.percent
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.io.sort.factor to 100 from MR setting mapreduce.task.io.sort.factor
2016-07-19 19:12:49,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.connect.timeout to 180000 from MR setting mapreduce.reduce.shuffle.connect.timeout
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.internal.sorter.class to org.apache.hadoop.util.QuickSort from MR setting map.sort.class
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.merge.progress.records to 10000 from MR setting mapreduce.task.merge.progress.records
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress to false from MR setting mapreduce.map.output.compress
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.sort.spill.percent to 0.7 from MR setting mapreduce.map.sort.spill.percent
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.ssl.enable to false from MR setting mapreduce.shuffle.ssl.enabled
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead to true from MR setting mapreduce.ifile.readahead
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.parallel.copies to 30 from MR setting mapreduce.reduce.shuffle.parallelcopies
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.ifile.readahead.bytes to 4194304 from MR setting mapreduce.ifile.readahead.bytes
2016-07-19 19:12:49,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.task.input.post-merge.buffer.percent to 0.0 from MR setting mapreduce.reduce.input.buffer.percent
2016-07-19 19:12:49,998 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.credentials.path to /hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens from MR setting mapreduce.job.credentials.binary
2016-07-19 19:12:49,998 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.shuffle.read.timeout to 180000 from MR setting mapreduce.reduce.shuffle.read.timeout
2016-07-19 19:12:49,998 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper - Setting tez.runtime.compress.codec to org.apache.hadoop.io.compress.DefaultCodec from MR setting mapreduce.map.output.compress.codec
2016-07-19 19:12:50,168 [PigTezLauncher-0] INFO org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job
2016-07-19 19:12:50,337 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.4.0.0-169, revision=3c1431f45faaca982ecc8dad13a107787b834696, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160210-0711 ]
2016-07-19 19:12:51,260 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-07-19 19:12:51,388 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.228.128:8050
2016-07-19 19:12:52,675 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Using org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager to manage Timeline ACLs
2016-07-19 19:12:52,859 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-07-19 19:12:52,889 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Session mode. Starting session.
2016-07-19 19:12:52,909 [PigTezLauncher-0] INFO org.apache.tez.common.security.TokenCache - Merging additional tokens from binary file, binaryFileName=/hadoop/yarn/local/usercache/maria_dev/appcache/application_1468940847411_0003/container_e16_1468940847411_0003_01_000004/container_tokens
2016-07-19 19:12:52,929 [PigTezLauncher-0] INFO org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.4.0.0-169/tez/tez.tar.gz
2016-07-19 19:12:53,335 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Tez system stage directory hdfs://sandbox.hortonworks.com:8020/tmp/yarn/staging/.tez/application_1468940847411_0006 doesn't exist and is created
2016-07-19 19:12:53,654 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Writing domains for appattempt_1468940847411_0006_000001 to /ats/active/application_1468940847411_0006/appattempt_1468940847411_0006_000001/domainlog-appattempt_1468940847411_0006_000001
2016-07-19 19:12:53,839 [PigTezLauncher-0] INFO org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager - Created Timeline Domain for History ACLs, domainId=Tez_ATS_application_1468940847411_0006
2016-07-19 19:12:58,719 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1468940847411_0006
2016-07-19 19:12:58,762 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - The url to track the Tez Session: http://sandbox.hortonworks.com:8088/proxy/application_1468940847411_0006/
2016-07-19 19:15:30,911 [PigTezLauncher-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Submitting DAG PigLatin:script.pig-0_scope-0
2016-07-19 19:15:30,912 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Submitting dag to TezSession, sessionName=PigLatin:script.pig, applicationId=application_1468940847411_0006, dagName=PigLatin:script.pig-0_scope-0, callerContext={ context=PIG, callerType=PIG_SCRIPT_ID, callerId=PIG-script.pig-3910d5d6-b7d6-4739-b8d6-a42d82bd50d8 }
2016-07-19 19:15:47,268 [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - Submitted dag to TezSession, sessionName=PigLatin:script.pig, applicationId=application_1468940847411_0006, dagName=PigLatin:script.pig-0_scope-0
2016-07-19 19:15:48,493 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-07-19 19:15:48,493 [PigTezLauncher-0] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at sandbox.hortonworks.com/192.168.228.128:8050
2016-07-19 19:15:48,500 [PigTezLauncher-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Submitted DAG PigLatin:script.pig-0_scope-0. Application id: application_1468940847411_0006
2016-07-19 19:15:49,393 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - HadoopJobId: job_1468940847411_0006
2016-07-19 19:16:29,510 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:16:49,505 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:17:09,506 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:17:29,506 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:17:49,506 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:18:09,507 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:18:29,508 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:18:49,508 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:19:09,509 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:19:29,509 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:19:49,509 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:20:09,509 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:20:43,063 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:21:03,062 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:21:23,063 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:21:43,064 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:22:03,063 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:22:23,064 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:22:43,065 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:23:03,064 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:23:23,065 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:23:43,066 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:24:03,066 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:24:23,066 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:24:43,067 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:25:03,067 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:25:23,067 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:25:43,068 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:26:16,042 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:26:28,084 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:26:48,084 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:27:08,084 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:27:28,085 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:27:48,086 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:28:08,086 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:28:28,087 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:28:48,088 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:29:08,089 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:29:29,003 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:29:49,004 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:30:09,005 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:30:29,005 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:30:49,006 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:31:09,005 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:31:29,006 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:31:49,006 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:32:09,006 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:32:29,007 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:32:49,007 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:33:09,007 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:33:32,135 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 2 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:34:05,752 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 5 Succeeded: 0 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:34:25,751 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 2 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:34:45,751 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 2 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:35:05,751 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 3 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:35:25,752 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 3 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:35:45,752 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 3 Running: 1 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:36:05,753 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:36:25,843 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:36:45,754 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:37:05,754 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:37:25,754 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:37:45,756 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null
2016-07-19 19:39:59,207 [PigTezLauncher-0] INFO org.apache.tez.common.counters.Limits - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=3000, COUNTER_NAME_MAX=64, MAX_COUNTERS=10000
2016-07-19 19:40:08,100 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:40:20,660 [PigTezLauncher-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:42:52,853 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:42:55,586 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:43:15,097 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:43:26,201 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:28,080 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:29,123 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:30,172 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:31,173 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:32,202 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:33,301 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:34,402 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:35,209 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:43:35,885 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:37,076 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:38,078 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 10 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:39,079 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 11 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:40,080 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 12 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:41,081 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 13 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:42,085 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 14 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:43,113 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 15 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:44,114 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 16 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:45,115 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 17 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:46,117 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 18 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:47,118 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 19 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:48,120 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 20 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:49,130 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 21 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:50,131 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 22 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:51,149 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 23 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:52,152 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 24 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:53,154 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 25 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:54,155 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 26 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:54,967 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:43:55,157 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 27 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:56,157 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 28 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:57,164 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 29 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:58,165 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 30 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:43:59,166 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 31 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:00,167 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 32 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:01,168 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 33 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:02,175 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 34 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:03,176 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 35 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:04,177 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 36 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:05,187 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 37 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:06,188 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 38 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:07,190 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 39 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:08,200 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 40 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:09,202 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 41 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:10,203 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 42 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:11,214 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 43 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:12,215 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 44 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:13,225 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 45 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:14,226 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 46 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:15,060 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:44:15,262 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 47 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:16,275 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 48 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:17,276 [PigTezLauncher-0] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: sandbox.hortonworks.com/192.168.228.128:52539. Already tried 49 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2016-07-19 19:44:17,592 [PigTezLauncher-0] WARN org.apache.pig.tools.pigstats.tez.TezDAGStats - Failed to get status for vertex: scope-54
2016-07-19 19:44:28,127 [PigTezLauncher-0] WARN org.apache.pig.tools.pigstats.tez.TezDAGStats - Failed to get status for vertex: scope-51
2016-07-19 19:44:35,282 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 4 Succeeded: 4 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 171
org.apache.tez.common.counters.DAGCounter
NUM_SUCCEEDED_TASKS=4
TOTAL_LAUNCHED_TASKS=4
DATA_LOCAL_TASKS=2
AM_CPU_MILLISECONDS=30700
AM_GC_TIME_MILLIS=12384
File System Counters
FILE_BYTES_READ=3397
FILE_BYTES_WRITTEN=2335
FILE_READ_OPS=0
FILE_LARGE_READ_OPS=0
FILE_WRITE_OPS=0
HDFS_BYTES_READ=25911
HDFS_BYTES_WRITTEN=1455
HDFS_READ_OPS=9
HDFS_LARGE_READ_OPS=0
HDFS_WRITE_OPS=2
org.apache.tez.common.counters.TaskCounter
REDUCE_INPUT_GROUPS=298
REDUCE_INPUT_RECORDS=298
COMBINE_INPUT_RECORDS=0
SPILLED_RECORDS=596
NUM_SHUFFLED_INPUTS=5
NUM_SKIPPED_INPUTS=0
NUM_FAILED_SHUFFLE_INPUTS=0
MERGED_MAP_OUTPUTS=5
GC_TIME_MILLIS=186160
CPU_MILLISECONDS=60430
PHYSICAL_MEMORY_BYTES=915931136
VIRTUAL_MEMORY_BYTES=3519447040
COMMITTED_HEAP_BYTES=915931136
INPUT_RECORDS_PROCESSED=8100
OUTPUT_RECORDS=755
OUTPUT_BYTES=7458
OUTPUT_BYTES_WITH_OVERHEAD=4265
OUTPUT_BYTES_PHYSICAL=2191
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILLS_BYTES_READ=2191
ADDITIONAL_SPILL_COUNT=0
SHUFFLE_CHUNK_COUNT=3
SHUFFLE_BYTES=2191
SHUFFLE_BYTES_DECOMPRESSED=4265
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_DISK_DIRECT=2191
NUM_MEM_TO_DISK_MERGES=0
NUM_DISK_TO_DISK_MERGES=0
SHUFFLE_PHASE_TIME=405
MERGE_PHASE_TIME=1339
FIRST_EVENT_RECEIVED=159
LAST_EVENT_RECEIVED=160
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_52_INPUT_scope_51
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_52
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
Shuffle Errors_scope_54_INPUT_scope_53
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
TaskCounter_scope_51_INPUT_scope_2
INPUT_RECORDS_PROCESSED=8000
TaskCounter_scope_51_OUTPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=4992
OUTPUT_BYTES_PHYSICAL=605
OUTPUT_BYTES_WITH_OVERHEAD=1377
OUTPUT_RECORDS=457
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_52_INPUT_scope_51
ADDITIONAL_SPILLS_BYTES_READ=605
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=106
LAST_EVENT_RECEIVED=106
MERGED_MAP_OUTPUTS=1
MERGE_PHASE_TIME=1241
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=1
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=605
SHUFFLE_BYTES_DECOMPRESSED=1377
SHUFFLE_BYTES_DISK_DIRECT=605
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=322
SPILLED_RECORDS=99
TaskCounter_scope_52_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1074
OUTPUT_BYTES_PHYSICAL=683
OUTPUT_BYTES_WITH_OVERHEAD=1284
OUTPUT_RECORDS=99
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=99
TaskCounter_scope_53_INPUT_scope_24
INPUT_RECORDS_PROCESSED=100
TaskCounter_scope_53_OUTPUT_scope_54
ADDITIONAL_SPILLS_BYTES_READ=0
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
ADDITIONAL_SPILL_COUNT=0
OUTPUT_BYTES=1392
OUTPUT_BYTES_PHYSICAL=903
OUTPUT_BYTES_WITH_OVERHEAD=1604
OUTPUT_RECORDS=100
SHUFFLE_CHUNK_COUNT=1
SPILLED_RECORDS=100
TaskCounter_scope_54_INPUT_scope_52
ADDITIONAL_SPILLS_BYTES_READ=683
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=28
LAST_EVENT_RECEIVED=28
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=43
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=99
REDUCE_INPUT_RECORDS=99
SHUFFLE_BYTES=683
SHUFFLE_BYTES_DECOMPRESSED=1284
SHUFFLE_BYTES_DISK_DIRECT=683
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=34
SPILLED_RECORDS=99
TaskCounter_scope_54_INPUT_scope_53
ADDITIONAL_SPILLS_BYTES_READ=903
ADDITIONAL_SPILLS_BYTES_WRITTEN=0
COMBINE_INPUT_RECORDS=0
FIRST_EVENT_RECEIVED=25
LAST_EVENT_RECEIVED=26
MERGED_MAP_OUTPUTS=2
MERGE_PHASE_TIME=55
NUM_DISK_TO_DISK_MERGES=0
NUM_FAILED_SHUFFLE_INPUTS=0
NUM_MEM_TO_DISK_MERGES=0
NUM_SHUFFLED_INPUTS=2
NUM_SKIPPED_INPUTS=0
REDUCE_INPUT_GROUPS=100
REDUCE_INPUT_RECORDS=100
SHUFFLE_BYTES=903
SHUFFLE_BYTES_DECOMPRESSED=1604
SHUFFLE_BYTES_DISK_DIRECT=903
SHUFFLE_BYTES_TO_DISK=0
SHUFFLE_BYTES_TO_MEM=0
SHUFFLE_PHASE_TIME=49
SPILLED_RECORDS=100
TaskCounter_scope_54_OUTPUT_scope_50
OUTPUT_RECORDS=99
org.apache.hadoop.mapreduce.TaskCounter
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_51_OUTPUT_scope_52
COMBINE_INPUT_RECORDS=99
COMBINE_OUTPUT_RECORDS=457
org.apache.hadoop.mapreduce.TaskCounter_scope_52_INPUT_scope_51
COMBINE_INPUT_RECORDS=0
COMBINE_OUTPUT_RECORDS=0
2016-07-19 19:44:41,372 [PigTezLauncher-0] WARN org.apache.pig.tools.pigstats.tez.TezDAGStats - Failed to get status for vertex: scope-53
2016-07-19 19:44:43,376 [PigTezLauncher-0] WARN org.apache.pig.tools.pigstats.tez.TezDAGStats - Failed to get status for vertex: scope-52
2016-07-19 19:44:43,946 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Counters are not available for vertex scope-51. Not computing warning aggregates.
2016-07-19 19:44:44,360 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Counters are not available for vertex scope-52. Not computing warning aggregates.
2016-07-19 19:44:44,366 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Counters are not available for vertex scope-53. Not computing warning aggregates.
2016-07-19 19:44:44,366 [main] WARN org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Counters are not available for vertex scope-54. Not computing warning aggregates.
2016-07-19 19:44:44,912 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.7.1.2.4.0.0-169
PigVersion: 0.15.0.2.4.0.0-169
TezVersion: 0.7.0.2.4.0.0-169
UserId: yarn
FileName: script.pig
StartedAt: 2016-07-19 19:12:33
FinishedAt: 2016-07-19 19:44:44
Features: HASH_JOIN,GROUP_BY,FILTER
Success!
DAG PigLatin:script.pig-0_scope-0:
ApplicationId: job_1468940847411_0006
TotalLaunchedTasks: 4
FileBytesRead: 3397
FileBytesWritten: 2335
HdfsBytesRead: 25911
HdfsBytesWritten: 1455
Input(s):
Output(s):
2016-07-19 19:44:48,509 [main] INFO org.apache.pig.Main - Pig script completed in 32 minutes, 54 seconds and 51 milliseconds (1974051 ms)
2016-07-19 19:44:48,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
2016-07-19 19:44:48,948 [Thread-34] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Tez session is already shutdown org.apache.tez.client.TezClient@167df515
2016-07-19 19:44:49,079 [Thread-1] ERROR org.apache.pig.impl.io.FileLocalizer - java.io.IOException: Filesystem closed