Hi,
I am running Pig action from Oozie workflow But Pig action ended with below error,So I tried changing yarn.timeline-service.version from 2.0f to 1.5f still it throws below error,
2019-12-06 13:54:21,074 [main] WARN org.apache.pig.PigServer - Error posting to ATS:
org.apache.hadoop.service.ServiceStateException: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x
at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at org.apache.pig.backend.hadoop.PigATSClient.<init>(PigATSClient.java:68)
at org.apache.pig.backend.hadoop.PigATSClient.getInstance(PigATSClient.java:57)
at org.apache.pig.PigServer.<init>(PigServer.java:260)
at org.apache.pig.PigServer.<init>(PigServer.java:219)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:46)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:273)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:216)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:78)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:231)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
... 24 more
Then I set yarn.timeline-service.enabled=false after setting this property above error gets resolve but Pig action stuck in running state.
Below is the pig job log:
2019-12-06 14:15:42,253 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2019-12-06 14:15:42,253 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1575641509651_0005]
Heart beat
Heart beat
Heart beat
Heart beat
2019-12-06 14:17:43,613 [HiveClientCache-cleaner-0] INFO org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a connection to metastore, current connections: 1
2019-12-06 14:17:43,613 [HiveClientCache-cleaner-0] INFO org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a connection to metastore, current connections: 0
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
End of LogType:stdout.This log file belongs to a running container (container_e24_1575641509651_0004_01_000002) and so may not be complete.
It seems like Pig job lost connection to the HiveMetaStoreClient but Hive services are up and running still it stuck in running state.
Thanks,
