Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Issue while running Pig job from Oozie : Conncetion lost to HiveMetaStoreClient

Highlighted

Issue while running Pig job from Oozie : Conncetion lost to HiveMetaStoreClient

New Contributor


Hi,

I am running Pig action from Oozie workflow But Pig action ended with below error,So I tried changing yarn.timeline-service.version from 2.0f to 1.5f still it throws below error,

 

 

2019-12-06 13:54:21,074 [main] WARN  org.apache.pig.PigServer  - Error posting to ATS:
org.apache.hadoop.service.ServiceStateException: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x
        at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
        at org.apache.pig.backend.hadoop.PigATSClient.<init>(PigATSClient.java:68)
        at org.apache.pig.backend.hadoop.PigATSClient.getInstance(PigATSClient.java:57)
        at org.apache.pig.PigServer.<init>(PigServer.java:260)
        at org.apache.pig.PigServer.<init>(PigServer.java:219)
        at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:46)
        at org.apache.pig.Main.run(Main.java:495)
        at org.apache.pig.PigRunner.run(PigRunner.java:49)
        at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:273)
        at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:216)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:78)
        at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:67)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:231)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x
        at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
        ... 24 more

 

 

Then I set yarn.timeline-service.enabled=false after setting this property above error gets resolve but Pig action stuck in running state.
Below is the pig job log:

 

 

2019-12-06 14:15:42,253 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
2019-12-06 14:15:42,253 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Running jobs are [job_1575641509651_0005]
Heart beat
Heart beat
Heart beat
Heart beat
2019-12-06 14:17:43,613 [HiveClientCache-cleaner-0] INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient  - Closed a connection to metastore, current connections: 1
2019-12-06 14:17:43,613 [HiveClientCache-cleaner-0] INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient  - Closed a connection to metastore, current connections: 0
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
End of LogType:stdout.This log file belongs to a running container (container_e24_1575641509651_0004_01_000002) and so may not be complete.

 

 


It seems like Pig job lost connection to the HiveMetaStoreClient but Hive services are up and running still it stuck in running state.

 

Thanks,

1 REPLY 1

Re: Issue while running Pig job from Oozie : Conncetion lost to HiveMetaStoreClient

New Contributor

I added piggybank.jar in Pig script,

    REGISTER hdfs://sandbox-hdp.hrotonworks.com:8020/lib/piggybank.jar;

I changed this path to local path :

   REGISTER /usr/hdp/3.0.1.0-78/pig/lib/piggybank.jar;

after that the issue is resolved.

Don't have an account?
Coming from Hortonworks? Activate your account here