Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

New Contributor

Atlas metadata is started but UI is not accessible :

Error log :

Exception in thread "main" java.lang.ExceptionInInitializerError at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115) at org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:148) at org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:95) at com.google.inject.AbstractModule.configure(AbstractModule.java:62) at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340) at com.google.inject.spi.Elements.getElements(Elements.java:110) at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104) at com.google.inject.Guice.createInjector(Guice.java:96) at com.google.inject.Guice.createInjector(Guice.java:84) at org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77) at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47) at org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.server.Server.start(Server.java:387) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) at org.eclipse.jetty.server.Server.doStart(Server.java:354) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:93) at org.apache.atlas.Atlas.main(Atlas.java:118) Caused by: com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1 at com.esotericsoftware.kryo.io.Input.require(Input.java:158) at com.esotericsoftware.kryo.io.Input.readVarInt(Input.java:355) at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:109) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:767) at com.thinkaurelius.titan.graphdb.database.serialize.kryo.KryoSerializer.readClassAndObject(KryoSerializer.java:94) at com.thinkaurelius.titan.graphdb.database.serialize.StandardSerializer.readClassAndObject(StandardSerializer.java:102) at com.thinkaurelius.titan.diskstorage.configuration.backend.KCVSConfiguration.staticBuffer2Object(KCVSConfiguration.java:252) at com.thinkaurelius.titan.diskstorage.configuration.backend.KCVSConfiguration.toMap(KCVSConfiguration.java:187) at com.thinkaurelius.titan.diskstorage.configuration.backend.KCVSConfiguration.asReadConfiguration(KCVSConfiguration.java:194) at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1383) at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93) at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73) at org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:105) at org.apache.atlas.repository.graph.GraphHelper.getInstance(GraphHelper.java:94) at org.apache.atlas.repository.graph.DeleteHandler.<clinit>(DeleteHandler.java:50) ... 32 more

8 REPLIES 8

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

New Contributor

Version HDP 2.5

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

Expert Contributor

@Sebastien WAFFLART Are you using the HBase as backend? Looking at the stack trace it appears that TitanDB is having trouble initializing.

During startup Atlas tries to populate the backend database with default type information. During this time, it tries to connect to the database. The failure seems to be during that time.

Can you please give more information about your environment? Do you have access to the atlas-application.properties file on the server?

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

New Contributor

Yes backend is HBASE 1.1.2 (HDP 2.5), REDHAT 7, security using Ranger is activated based on OpenLDAP.

14451-atlas.png

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

New Contributor

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

Expert Contributor

@Sebastien WAFFLART This is still being investigated.

Highlighted

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

@Sebastien WAFFLART

Atlas uses titan as graphDB and titan in-turn uses apache tinkerpop for graph computing framework to query and store graph databases. The issue is coming from tinkerpop configuration where the default buffersize for the query response is configured to 4096.

If the response for the query is greater than 4096 bytes then it results in `KryoException` errors that complain of "Buffer too small". This issue can be fixed by configuring the buffer size value to desirable number, which is available in >= tinkerpop-3.0.2-incubating version. You can find more information about this issue at TINKERPOP-817 and documentation here.

In HDP-2.5 release, atlas uses tinkerpop version 2.6.0 and hence it manifests..

I have created bug to track and fix this in atlas. (https://issues.apache.org/jira/browse/ATLAS-1725).

Thanks for bringing this to our notice.

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

Contributor

@Sebastien WAFFLART - this error seems to be hit during Atlas startup. And value of capacity/required in the error message are too small - 0 and 1: "com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1".

I guess something else might be going on here. Can you attach Atlas log file to troubleshoot this further?

Re: Atlas Metadata - com.esotericsoftware.kryo.KryoException: Buffer too small: capacity: 0, required: 1

New Contributor