Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error while trying to store pig output to the Hive Table ?

Error while trying to store pig output to the Hive Table ?

Expert Contributor

I created the Hive Table under database testdb:

create table tbl2 (year string, DeptTime string, ArrTime string) stored as ORC;

after that i created the following Pig Script:

A = LOAD '/test/input/' USING PigStorage(',') AS (year:chararray,ArrTime:chararray,DeptTime:chararray);

describe A;

dump A;

while executing the above script it works fine.

But adding following line to store pig output to hive it will throw an error.

A = LOAD '/test/input/' USING PigStorage(',') AS (year:chararray,ArrTime:chararray,DeptTime:chararray);

describe A;

dump A;

store A into 'testdb.tbl2' using org.apache.hive.hcatalog.pig.HCatStorer();

used PIG Argument -useHCatalog

Error:

ls: cannot access '/mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493274253049_0026/container_1493274253049_0026_01_000002/hive.tar.gz/hive/lib/slf4j-api-*.jar': No such file or directory ls: cannot access '/mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493274253049_0026/container_1493274253049_0026_01_000002/hive.tar.gz/hive/hcatalog/lib/*hbase-storage-handler-*.jar':

2 REPLIES 2
Highlighted

Re: Error while trying to store pig output to the Hive Table ?

Expert Contributor

Is there more output to this? Perhaps a line with timestamp and ERROR? If so, can you include it here?

Re: Error while trying to store pig output to the Hive Table ?

Expert Contributor
ls: cannot access '/mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493443427239_0034/container_1493443427239_0034_01_000002/hive.tar.gz/hive/lib/slf4j-api-*.jar': No such file or directory 
 ls: cannot access '/mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493443427239_0034/container_1493443427239_0034_01_000002/hive.tar.gz/hive/hcatalog/lib/*hbase-storage-handler-*.jar': No such file or directory 
 WARNING: Use "yarn jar" to launch YARN applications. 
 17/04/29 07:47:53 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 
 17/04/29 07:47:53 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 
 17/04/29 07:47:53 INFO pig.ExecTypeProvider: Trying ExecType : TEZ_LOCAL 
 17/04/29 07:47:53 INFO pig.ExecTypeProvider: Trying ExecType : TEZ 
 17/04/29 07:47:53 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType 
 2017-04-29 07:47:53,318 [main] INFO  org.apache.pig.Main - Apache Pig version 0.16.0.2.5.4.0-121 (rexported) compiled Jan 23 2017, 18:00:55 
 2017-04-29 07:47:53,318 [main] INFO  org.apache.pig.Main - Logging error messages to: /mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493443427239_0034/container_1493443427239_0034_01_000002/pig_1493452073315.log 
 2017-04-29 07:47:53,851 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found 
 2017-04-29 07:47:53,949 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: wasb://hdpcluster@hdpstoragesa.blob.core.windows.net 
 2017-04-29 07:47:54,414 [main] INFO  org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-c57be107-d6ca-49cb-bb5b-13a96fcf5fd1 
 2017-04-29 07:47:54,800 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://headnodehost:8188/ws/v1/timeline/ 
 2017-04-29 07:47:54,933 [main] INFO  org.apache.pig.backend.hadoop.PigATSClient - Created ATS Hook 
 2017-04-29 07:47:55,844 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 
 2017-04-29 07:47:55,882 [main] INFO  org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code. 
 2017-04-29 07:47:55,915 [main] INFO  org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]} 
 2017-04-29 07:47:55,973 [main] INFO  org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (PS Old Gen) of size 716177408 to monitor. collectionUsageThreshold = 501324192, usageThreshold = 501324192 
 2017-04-29 07:47:56,101 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Tez staging directory is /tezstaging/yarn/staging and resources directory is /tmp/temp258650981 
 2017-04-29 07:47:56,131 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler - File concatenation threshold: 100 optimistic? false 
 2017-04-29 07:47:56,216 [main] INFO  org.apache.pig.builtin.PigStorage - Using PigTextInputFormat 
 2017-04-29 07:47:56,227 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 
 2017-04-29 07:47:56,228 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 
 2017-04-29 07:47:56,239 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libfb303-0.9.3.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: libthrift-0.9.3.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: jdo-api-3.0.1.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: pig-0.16.0.2.5.4.0-121-core-h2.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: automaton-1.11-8.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-exec-1.2.1000.2.5.4.0-121.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-metastore-1.2.1000.2.5.4.0-121.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hbase-handler-1.2.1000.2.5.4.0-121.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-pig-adapter-1.2.1000.2.5.4.0-121.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: antlr-runtime-3.4.jar 
 2017-04-29 07:47:57,686 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Local resource: hive-hcatalog-core-1.2.1000.2.5.4.0-121.jar 
 2017-04-29 07:47:57,886 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - For vertex - scope-12: parallelism=1, memory=1536, java opts=-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC 
 2017-04-29 07:47:57,886 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Processing aliases: A 
 2017-04-29 07:47:57,886 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Detailed locations: A[1,4],A[-1,-1] 
 2017-04-29 07:47:57,886 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder - Pig features in the vertex:  
 2017-04-29 07:47:57,959 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler - Total estimated parallelism is 1 
 2017-04-29 07:47:58,028 [PigTezLauncher-0] INFO  org.apache.pig.tools.pigstats.tez.TezScriptState - Pig script settings are added to the job 
 2017-04-29 07:47:58,047 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, version=0.7.0.2.5.4.0-121, revision=97277df8d4c60b31173b80b5c6110075b03f86a3, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2017-01-23T17:15:34Z ] 
 2017-04-29 07:47:58,159 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://headnodehost:8188/ws/v1/timeline/ 
 2017-04-29 07:47:58,294 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at headnodehost/10.0.0.18:10200 
 2017-04-29 07:47:58,327 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Using org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager to manage Timeline ACLs 
 2017-04-29 07:47:58,389 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://headnodehost:8188/ws/v1/timeline/ 
 2017-04-29 07:47:58,419 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Session mode. Starting session. 
 2017-04-29 07:47:58,427 [PigTezLauncher-0] INFO  org.apache.tez.common.security.TokenCache - Merging additional tokens from binary file, binaryFileName=/mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493443427239_0034/container_1493443427239_0034_01_000002/container_tokens 
 2017-04-29 07:47:58,428 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClientUtils - Using tez.lib.uris value from configuration: /hdp/apps/2.5.4.0-121/tez/tez.tar.gz 
 2017-04-29 07:47:58,562 [PigTezLauncher-0] WARN  org.apache.hadoop.ipc.Client - Failed to connect to server: hn0-hdpclu.hmh4knlgviduxjkeuokrymvbeg.ix.internal.cloudapp.net/10.0.0.18:8050: retries get failed due to exceeded maximum allowed retries number: 0 
 java.net.ConnectException: Connection refused 
 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 
 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) 
 	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) 
 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) 
 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) 
 	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:650) 
 	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:745) 
 	at org.apache.hadoop.ipc.Client$Connection.access$3200(Client.java:397) 
 	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1618) 
 	at org.apache.hadoop.ipc.Client.call(Client.java:1449) 
 	at org.apache.hadoop.ipc.Client.call(Client.java:1396) 
 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) 
 	at com.sun.proxy.$Proxy25.getNewApplication(Unknown Source) 
 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getNewApplication(ApplicationClientProtocolPBClientImpl.java:221) 
 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
 	at java.lang.reflect.Method.invoke(Method.java:498) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:278) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176) 
 	at com.sun.proxy.$Proxy26.getNewApplication(Unknown Source) 
 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNewApplication(YarnClientImpl.java:225) 
 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createApplication(YarnClientImpl.java:233) 
 	at org.apache.tez.client.TezYarnClient.createApplication(TezYarnClient.java:66) 
 	at org.apache.tez.client.TezClient.createApplication(TezClient.java:901) 
 	at org.apache.tez.client.TezClient.start(TezClient.java:417) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:100) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:232) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:203) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) 
 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 	at java.lang.Thread.run(Thread.java:745) 
 2017-04-29 07:47:58,656 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider - Failing over to rm2 
 2017-04-29 07:47:58,846 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Tez system stage directory wasb://hdpcluster@hdpstoragesa.blob.core.windows.net/tezstaging/yarn/staging/.tez/application_1493443427239_0035 doesn't exist and is created 
 2017-04-29 07:47:59,012 [PigTezLauncher-0] INFO  org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager - Created Timeline Domain for History ACLs, domainId=Tez_ATS_application_1493443427239_0035 
 2017-04-29 07:47:59,442 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1493443427239_0035 
 2017-04-29 07:47:59,445 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - The url to track the Tez Session: http://hn1-hdpclu.hmh4knlgviduxjkeuokrymvbeg.ix.internal.cloudapp.net:8088/proxy/application_1493443... 
 2017-04-29 07:48:04,578 [PigTezLauncher-0] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Submitting DAG PigLatin:script.pig-0_scope-0 
 2017-04-29 07:48:04,578 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Submitting dag to TezSession, sessionName=PigLatin:script.pig, applicationId=application_1493443427239_0035, dagName=PigLatin:script.pig-0_scope-0, callerContext={ context=PIG, callerType=PIG_SCRIPT_ID, callerId=PIG-script.pig-c57be107-d6ca-49cb-bb5b-13a96fcf5fd1 } 
 2017-04-29 07:48:04,951 [PigTezLauncher-0] INFO  org.apache.tez.client.TezClient - Submitted dag to TezSession, sessionName=PigLatin:script.pig, applicationId=application_1493443427239_0035, dagName=PigLatin:script.pig-0_scope-0 
 2017-04-29 07:48:05,149 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://headnodehost:8188/ws/v1/timeline/ 
 2017-04-29 07:48:05,152 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at headnodehost/10.0.0.18:10200 
 2017-04-29 07:48:05,190 [PigTezLauncher-0] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Submitted DAG PigLatin:script.pig-0_scope-0. Application id: application_1493443427239_0035 
 2017-04-29 07:48:05,194 [PigTezLauncher-0] WARN  org.apache.hadoop.ipc.Client - Failed to connect to server: hn0-hdpclu.hmh4knlgviduxjkeuokrymvbeg.ix.internal.cloudapp.net/10.0.0.18:8050: retries get failed due to exceeded maximum allowed retries number: 0 
 java.net.ConnectException: Connection refused 
 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 
 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) 
 	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) 
 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) 
 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) 
 	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:650) 
 	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:745) 
 	at org.apache.hadoop.ipc.Client$Connection.access$3200(Client.java:397) 
 	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1618) 
 	at org.apache.hadoop.ipc.Client.call(Client.java:1449) 
 	at org.apache.hadoop.ipc.Client.call(Client.java:1396) 
 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) 
 	at com.sun.proxy.$Proxy25.getApplicationReport(Unknown Source) 
 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplicationReport(ApplicationClientProtocolPBClientImpl.java:191) 
 	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) 
 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
 	at java.lang.reflect.Method.invoke(Method.java:498) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:278) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194) 
 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176) 
 	at com.sun.proxy.$Proxy26.getApplicationReport(Unknown Source) 
 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:436) 
 	at org.apache.tez.client.TezYarnClient.getApplicationReport(TezYarnClient.java:89) 
 	at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.getAppReport(DAGClientRPCImpl.java:231) 
 	at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.createAMProxyIfNeeded(DAGClientRPCImpl.java:251) 
 	at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.getDAGStatus(DAGClientRPCImpl.java:96) 
 	at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatusViaAM(DAGClientImpl.java:367) 
 	at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatusInternal(DAGClientImpl.java:227) 
 	at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatus(DAGClientImpl.java:275) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:224) 
 	at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:210) 
 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 	at java.lang.Thread.run(Thread.java:745) 
 2017-04-29 07:48:05,200 [PigTezLauncher-0] INFO  org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider - Failing over to rm2 
 2017-04-29 07:48:06,001 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - HadoopJobId: job_1493443427239_0035 
 2017-04-29 07:48:06,191 [Timer-0] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=null 
 2017-04-29 07:48:09,462 [PigTezLauncher-0] INFO  org.apache.tez.common.counters.Limits - Counter limits initialized with parameters:  GROUP_NAME_MAX=256, MAX_GROUPS=3000, COUNTER_NAME_MAX=64, MAX_COUNTERS=10000 
 2017-04-29 07:48:09,465 [PigTezLauncher-0] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=SUCCEEDED, progress=TotalTasks: 1 Succeeded: 1 Running: 0 Failed: 0 Killed: 0, diagnostics=, counters=Counters: 18 
 	org.apache.tez.common.counters.DAGCounter 
 		NUM_SUCCEEDED_TASKS=1 
 		TOTAL_LAUNCHED_TASKS=1 
 		RACK_LOCAL_TASKS=1 
 		AM_CPU_MILLISECONDS=1560 
 		AM_GC_TIME_MILLIS=78 
 	File System Counters 
 		WASB_BYTES_READ=163 
 		WASB_BYTES_WRITTEN=253 
 	org.apache.tez.common.counters.TaskCounter 
 		GC_TIME_MILLIS=56 
 		CPU_MILLISECONDS=2520 
 		PHYSICAL_MEMORY_BYTES=225968128 
 		VIRTUAL_MEMORY_BYTES=3024261120 
 		COMMITTED_HEAP_BYTES=225968128 
 		INPUT_RECORDS_PROCESSED=10 
 		INPUT_SPLIT_LENGTH_BYTES=163 
 		OUTPUT_RECORDS=10 
 	TaskCounter_scope_12_INPUT_scope_0 
 		INPUT_RECORDS_PROCESSED=10 
 		INPUT_SPLIT_LENGTH_BYTES=163 
 	TaskCounter_scope_12_OUTPUT_scope_11 
 		OUTPUT_RECORDS=10 
 2017-04-29 07:48:09,510 [PigTezLauncher-0] INFO  org.apache.pig.tools.pigstats.JobStats - using output size reader: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader 
 2017-04-29 07:48:10,008 [main] INFO  org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: 
        HadoopVersion: 2.7.3.2.5.4.0-121                                                                                    
           PigVersion: 0.16.0.2.5.4.0-121                                                                                   
           TezVersion: 0.7.0.2.5.4.0-121                                                                                    
               UserId: yarn                                                                                                 
             FileName: script.pig                                                                                           
            StartedAt: 2017-04-29 07:47:56                                                                                  
           FinishedAt: 2017-04-29 07:48:10                                                                                  
             Features: UNKNOWN                                                                                              
 Success! 
 DAG 0: 
                                     Name: PigLatin:script.pig-0_scope-0                                                                        
                            ApplicationId: job_1493443427239_0035                                                                               
                       TotalLaunchedTasks: 1                                                                                                    
                            FileBytesRead: 0                                                                                                    
                         FileBytesWritten: 0                                                                                                    
                            HdfsBytesRead: 0                                                                                                    
                         HdfsBytesWritten: 0                                                                                                    
       SpillableMemoryManager spill count: 0                                                                                                    
                 Bags proactively spilled: 0                                                                                                    
              Records proactively spilled: 0                                                                                                    
 DAG Plan: 
 Tez vertex scope-12 
 Vertex Stats: 
 VertexId Parallelism TotalTasks   InputRecords   ReduceInputRecords  OutputRecords  FileBytesRead FileBytesWritten  HdfsBytesRead HdfsBytesWritten Alias	Feature	Outputs 
 scope-12           1          1             10                    0             10              0                0              0                0 A		wasb://hdpcluster@hdpstoragesa.blob.core.windows.net/tmp/temp2140435255/tmp1840093578, 
 Input(s): 
 Successfully read 10 records from: "/test/input/t12007.csv" 
 Output(s): 
 Successfully stored 10 records (253 bytes) in: "wasb://hdpcluster@hdpstoragesa.blob.core.windows.net/tmp/temp2140435255/tmp1840093578" 
 2017-04-29 07:48:10,039 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 
 2017-04-29 07:48:10,039 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 
 2017-04-29 07:48:10,363 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist 
 2017-04-29 07:48:10,363 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.root.logger does not exist 
 2017-04-29 07:48:10,397 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://hn1-hdpclu.hmh4knlgviduxjkeuokrymvbeg.ix.internal.cloudapp.net:9083 
 2017-04-29 07:48:10,415 [main] INFO  hive.metastore - Connected to metastore. 
 2017-04-29 07:48:11,184 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Column names should all be in lowercase. Invalid name found: ArrTime 
 Details at logfile: /mnt/resource/hadoop/yarn/local/usercache/admin/appcache/application_1493443427239_0034/container_1493443427239_0034_01_000002/pig_1493452073315.log 
 2017-04-29 07:48:11,198 [main] INFO  org.apache.pig.Main - Pig script completed in 18 seconds and 30 milliseconds (18030 ms) 
 2017-04-29 07:48:11,221 [main] WARN  org.apache.hadoop.fs.azure.AzureFileSystemThreadPoolExecutor - Disabling threads for Delete operation as thread count 0 is <= 1 
 2017-04-29 07:48:11,247 [main] INFO  org.apache.hadoop.fs.azure.AzureFileSystemThreadPoolExecutor - Time taken for Delete operation is: 27 ms with threads: 0 
 2017-04-29 07:48:11,271 [main] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool 
 2017-04-29 07:48:11,294 [pool-1-thread-1] INFO  org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - Shutting down Tez session org.apache.tez.client.TezClient@395a9811 
 2017-04-29 07:48:11,306 [pool-1-thread-1] INFO  org.apache.tez.client.TezClient - Shutting down Tez Session, sessionName=PigLatin:script.pig, applicationId=application_1493443427239_0035 
 2017-04-29 07:48:11,363 [pool-1-thread-1] WARN  org.apache.hadoop.fs.azure.AzureFileSystemThreadPoolExecutor - Disabling threads for Delete operation as thread count 0 is <= 1 

2017-04-29 07:48:11,439 [pool-1-thread-1] INFO org.apache.hadoop.fs.azure.AzureFileSystemThreadPoolExecutor - Time taken for Delete operation is: 76 ms with threads: 0