Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Object with id "" is managed by a different Object Manager

Object with id "" is managed by a different Object Manager

Explorer

Hello,

 

from time to time I am running into a strange problem.

When I issue a command in the hive shell I get the error

 

hive> show databases;
FAILED: Error in metadata: MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException javax.jdo.JDOUserException: Object with id "" is managed by a different Object Manager
NestedThrowables:
org.datanucleus.exceptions.NucleusUserException: Object with id "" is managed by a different Object Manager)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

 

I do not know what user this message is refering to.

I started the hive shell as a different user but got the same result.

The only way to get rid of this error is to retart the complete cloudera cluster.

Is there any other way to get rid of this error or - even better - prevent it?

 

I must add that I am accessing the databas ewith both hive and impala - could that be a problem?

 

3 REPLIES 3

Re: Object with id "" is managed by a different Object Manager

New Contributor

I am seeing this error also with HiveServer2. Hive will run fine for a couple of weeks and then starting throwing these errors for "LOAD DATA LOCAL INPATH" commands.... Throws for about 2 hours and then bam! starts working again....

 

2014-03-16 15:11:44,880 INFO hive.ql.parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH '/talend_work/distcheckin_temp/PhoneCheckIn_bulk_6101740.txt' INTO TABLE phonecheckin_staging_talend PARTITION (batchid='c213eea7-ad3e-11e3-9416-fa8ac334b8fe' ,jobcycleid=6101740)
2014-03-16 15:11:44,880 INFO hive.ql.parse.ParseDriver: Parse Completed
2014-03-16 15:11:44,880 INFO org.apache.hadoop.hive.ql.exec.Task: Copying data from file:/talend_work/distcheckin_temp/PhoneCheckIn_bulk_6101743.txt to hdfs://rep-hadoop1.mp.corp:8020/tmp/hive-hive/hive_2014-03-16_15-11-40_668_1274510217446735506-12/-ext-10000
2014-03-16 15:11:45,883 ERROR org.apache.hadoop.hive.ql.Driver: FAILED: SemanticException Line 1:96 Exception while processing 'phonecheckin_staging_talend': Unable to fetch table phonecheckin_staging_talend
org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:96 Exception while processing 'phonecheckin_staging_talend': Unable to fetch table phonecheckin_staging_talend
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer$tableSpec.<init>(BaseSemanticAnalyzer.java:685)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer$tableSpec.<init>(BaseSemanticAnalyzer.java:656)
at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:201)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:355)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:82)
at org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:129)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:209)
at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:154)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:208)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table phonecheckin_staging_talend
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:936)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:870)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer$tableSpec.<init>(BaseSemanticAnalyzer.java:679)
... 21 more
Caused by: MetaException(message:javax.jdo.JDOUserException: Object with id "" is managed by a different Object Manager
NestedThrowables:
org.datanucleus.exceptions.NucleusUserException: Object with id "" is managed by a different Object Manager)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:25234)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:25211)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:25142)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:879)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:865)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:780)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy9.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:928)
... 23 more

Re: Object with id "" is managed by a different Object Manager

New Contributor
We get something similar recently in our METASTORE logs on hive jobs started from Oozie and failing. Anyone have ideas? CHD 4.5

2014-03-26 08:23:09,779 INFO DataNucleus.JDO: Exception thrown
Object with id "" is managed by a different Object Manager
org.datanucleus.exceptions.NucleusUserException: Object with id "" is managed by a different Object Manager
at org.datanucleus.ExecutionContextImpl.detachObject(ExecutionContextImpl.java:2735)
at org.datanucleus.ExecutionContextThreadedImpl.detachObject(ExecutionContextThreadedImpl.java:329)
at org.datanucleus.store.types.SCOUtils.detachForCollection(SCOUtils.java:1251)
at org.datanucleus.store.fieldmanager.DetachFieldManager.internalFetchObjectField(DetachFieldManager.java:177)
at org.datanucleus.store.fieldmanager.AbstractFetchDepthFieldManager.fetchObjectField(AbstractFetchDepthFieldManager.java:113)
at org.datanucleus.state.JDOStateManager.detach(JDOStateManager.java:3988)
at org.datanucleus.ExecutionContextImpl.performDetachAllOnTxnEnd(ExecutionContextImpl.java:4504)
at org.datanucleus.ExecutionContextImpl.postCommit(ExecutionContextImpl.java:4603)
at org.datanucleus.ExecutionContextImpl.transactionCommitted(ExecutionContextImpl.java:620)
at org.datanucleus.TransactionImpl.internalPostCommit(TransactionImpl.java:550)
at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:331)
at org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:98)
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:345)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy7.commitTransaction(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:185) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table(HiveMetaStore.java:2304)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table(HiveMetaStore.java:2276)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105) at $Proxy8.alter_table(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_table.getResult(ThriftHiveMetastore.java:6047)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_table.getResult(ThriftHiveMetastore.java:6031) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:587)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:584)

Re: Object with id "" is managed by a different Object Manager

New Contributor

When we ran into the issue I noticed that we did not have Zookeeper installed/running. Instructed my sys admin to install zookeeper and the problem went away.