Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas hard delete bug

Highlighted

Atlas hard delete bug

New Contributor

hello there,

My team is developing a metadata system based on Atlas. And when we tried to hard delete metadata, it fails because current logic is like this:

1. hard delete entity vertices;

2. notify listeners in which trying to find ITypeReferenceable by entity id while the entity is hard deleted already.

3. notify fails and throws Exception.

To enable hard delete, I config the "atlas.DeleteHandlerV1.impl =org.apache.atlas.repository.store.graph.v1.HardDeleteHandlerV1"

The api I used is "DELETE v2/entity/guid/{guid}"

And the error log is as below:

2017-08-08 16:51:40,917 INFO - [pool-1-thread-41 - 56e5790a-ebea-4042-917c-1a2627bb6057:] ~ GraphTransaction intercept for org.apache.

atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteById (GraphTransactionAdvisor$1:41)

2017-08-08 16:51:40,934 INFO - [pool-1-thread-41 - 56e5790a-ebea-4042-917c-1a2627bb6057:] ~ GraphTransaction intercept for org.apache.

atlas.repository.graph.GraphBackedMetadataRepository.getEntityDefinition (GraphTransactionAdvisor$1:41)

2017-08-08 16:51:40,942 ERROR - [pool-1-thread-41 - 56e5790a-ebea-4042-917c-1a2627bb6057:] ~ graph rollback due to exception EntityNotF

oundException:Could not find entities in the repository with guids: [973d70ee-bc18-4085-89d0-30bff615918e] (GraphTransactionInterceptor

:73)

2017-08-08 16:51:40,944 ERROR - [pool-1-thread-41 - 56e5790a-ebea-4042-917c-1a2627bb6057:] ~ Exception while getting a typed reference

for the entity (AtlasInstanceConverter:114)

org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find entities in the repository with guids: [973d70ee-bc18-408

5-89d0-30bff615918e]

at org.apache.atlas.repository.graph.GraphBackedMetadataRepository.getEntityDefinitions(GraphBackedMetadataRepository.java:221)

at org.apache.atlas.repository.graph.GraphBackedMetadataRepository.getEntityDefinition(GraphBackedMetadataRepository.java:173)

at org.apache.atlas.repository.graph.GraphBackedMetadataRepository$$FastClassBySpringCGLIB$$43a564a2.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:60)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)

at org.apache.atlas.repository.graph.GraphBackedMetadataRepository$$EnhancerBySpringCGLIB$$bef95bd0.getEntityDefinition(<genera

ted>)

at org.apache.atlas.services.DefaultMetadataService.getEntityDefinition(DefaultMetadataService.java:329)

at org.apache.atlas.repository.converters.AtlasInstanceConverter.getITypedReferenceable(AtlasInstanceConverter.java:112)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.toITypedReferenceable(AtlasEntityChangeNotifier.java:18

4)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.notifyListeners(AtlasEntityChangeNotifier.java:158)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityChangeNotifier.onEntitiesMutated(AtlasEntityChangeNotifier.java:86)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteById(AtlasEntityStoreV1.java:375)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)

...skipping one line

at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:60)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)

at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$EnhancerBySpringCGLIB$$11ec0125.deleteById(<generated>)

at org.apache.atlas.web.rest.EntityREST.deleteByGuid(EntityREST.java:236)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)

at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractRes

ourceMethodDispatchProvider.java:185)

at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)

at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)

at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)

at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)

at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)

at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)

at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)

at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)

at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:76)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)

at org.apache.atlas.web.filters.AtlasAuthorizationFilter.doFilter(AtlasAuthorizationFilter.java:157)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

...skipping one line

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:11

1)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.apache.atlas.web.filters.AtlasCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(AtlasCSRFPreventionFilter.java:2

35)

at org.apache.atlas.web.filters.AtlasCSRFPreventionFilter.handleHttpInteraction(AtlasCSRFPreventionFilter.java:177)

at org.apache.atlas.web.filters.AtlasCSRFPreventionFilter.doFilter(AtlasCSRFPreventionFilter.java:190)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.apache.atlas.web.filters.AtlasAuthenticationFilter.doFilter(AtlasAuthenticationFilter.java:325)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareReque

stFilter.java:170)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:132)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.jav

a:215)

at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProces

singFilter.java:200)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105

)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

...skipping one line

at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)

at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)

at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)

at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:499)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)

at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

2017-08-08 16:51:40,947 ERROR - [pool-1-thread-41 - 56e5790a-ebea-4042-917c-1a2627bb6057:] ~ graph rollback due to exception AtlasBaseE

xception:Given instance is invalid/not found: Could not find entities in the repository with guids: [973d70ee-bc18-4085-89d0-30bff61591

8e] (GraphTransactionInterceptor:73)

2 REPLIES 2

Re: Atlas hard delete bug

New Contributor

We are trying something similar as well i.e. Create a custom entity to store information on data stored in S3. Once the hard delete is enabled, we can no longer able to delete or update the entity. Getting the error "Given instance is invalid/not found: Could not find entities in the repository with guids" Were you able to find any resolution?

HDP Version: 2.6.1

Atlas Version: 0.8.0

Re: Atlas hard delete bug

Contributor

@Petrina Zheng, @Jayaraman Palaniappan - the failure in hard-delete was due to ATLAS-2169. Fix for this issue will be included in the next HDP 2.6-maint release (2.6.5). However, please note that hard-delete is not a recommended configuration for production deployment - given Atlas's role in governance domain.

Don't have an account?
Coming from Hortonworks? Activate your account here