Schema registry breaks after upgrade to HDF 3.4.1.1

azure — Tue, 30 Jul 2019 19:23:04 GMT

Hi ,

We recently upgrade our DEV and PROD environments from HDF 3.3 to HDF 3.4.1.1 after following upgrade instructions on a HDF-only cluster. After the upgrade, schema registry is giving the following exceptions in both environments for clients using the confluent API:

ERROR [2019-07-30 11:17:42.014] [dw-94 - POST /api/v1/confluent/subjects/EgressLiveAllCXXX1-value/versions] c.h.r.s.w.ConfluentSchemaRegistryCompatibleResource - Encountered error while adding subject [EgressLiveAllCowBehavior1-value]

java.lang.RuntimeException: Failed to obtain write lock : schema_metadata_info-EgressLiveAllCXXXX1-value in 120 sec

at com.hortonworks.registries.schemaregistry.DefaultSchemaRegistry.lockSchemaMetadata(DefaultSchemaRegistry.java:520)

at com.hortonworks.registries.schemaregistry.DefaultSchemaRegistry.addSchemaVersion(DefaultSchemaRegistry.java:489)

at com.hortonworks.registries.schemaregistry.webservice.ConfluentSchemaRegistryCompatibleResource.lambda$registerSchemaVersion$1(ConfluentSchemaRegistryCompatibleResource.java:322)

at com.hortonworks.registries.schemaregistry.webservice.BaseRegistryResource.handleLeaderAction(BaseRegistryResource.java:77)

at com.hortonworks.registries.schemaregistry.webservice.ConfluentSchemaRegistryCompatibleResource.registerSchemaVersion(ConfluentSchemaRegistryCompatibleResource.java:307)

at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)

at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160)

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)

......

....

I've tried flusing the mySQL tables, restarted the database and the SR services but no luck

As this is impacting production, any help would be highly appreciated.

Re: Schema registry breaks after upgrade to HDF 3.4.1.1

azure — Tue, 30 Jul 2019 21:55:23 GMT

Turns out , the following needs to be executed on mySQL:

INSERT INTO schema_lock SELECT CONCAT('schema_metadata_info-', schema_metadata_info.name) AS name, schema_metadata_info.timestamp FROM schema_metadata_info WHERE CONCAT('schema_metadata_info-', schema_metadata_info.name) NOT IN (SELECT name FROM schema_lock);

Re: Schema registry breaks after upgrade to HDF 3.4.1.1

Thomas-Kopp — Tue, 04 Feb 2020 11:02:59 GMT

I was facing the same issue after updating to HDF 3.4.1.1, couldn't update/create new version of an existing schema. The solution worked out for me as well. I first checked whether the schema in question has a record in schema_lock. It hadn't. As well as most of the schemas in our registry.

Thank you very much for sharing!

question Schema registry breaks after upgrade to HDF 3.4.1.1 in Support Questions

Schema registry breaks after upgrade to HDF 3.4.1.1

Re: Schema registry breaks after upgrade to HDF 3.4.1.1

Re: Schema registry breaks after upgrade to HDF 3.4.1.1