Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

LinkageError: ClassCastException Hortonworks Schema Registry Client

LinkageError: ClassCastException Hortonworks Schema Registry Client

Explorer

I am getting the following error when launching a flink application that tries to read from schema registry:

 

 

I need to export some hadoop related directories to the classpath before launching the application to make it work (as in every maven application), and I am getting the following error.

 

Spoiler
> Caused by: java.lang.LinkageError: ClassCastException: attempting to
> castjar:file:/usr/hdp/3.0.0.0-1634/hadoop/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class
> to
> jar:file:/tmp/blobStore-634df1c1-ffc8-4610-86af-8f39b33e4250/job_ac11246bea2bb31008c1a78212357514/blob_p-79f2d3193313ea987c15b4b28411db0fc2aa436c-f858cb54126b6d546c01e5ed453bf106!/javax/ws/rs/ext/RuntimeDelegate.class
> at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:146)
> at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120)
> at javax.ws.rs.core.UriBuilder.newInstance(UriBuilder.java:95)
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
> at org.glassfish.jersey.client.JerseyWebTarget.<init>(JerseyWebTarget.java:71)
> at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:290)
> at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:76)
> at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.lambda$currentSchemaRegistryTargets$0(SchemaRegistryClient.java:293)
> at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
> at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.currentSchemaRegistryTargets(SchemaRegistryClient.java:293)
> at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getSupportedSchemaProviders(SchemaRegistryClient.java:384)
> at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getDefaultDeserializer(SchemaRegistryClient.java:969)
> at SchemaService.InitDeserializer(SchemaService.java:47)
> at SchemaService.deserialize(SchemaService.java:38)
> at org.apache.flink.streaming.connectors.kafka.internals.KafkaDeserializationSchemaWrapper.deserialize(KafkaDeserializationSchemaWrapper.java:45)
> at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:140)
> at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:712)
> at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)
> at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)
> at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)
> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:745)

 

After making some research, I figured out that the class the program is trying to use is present in two diferent JAR files:

 

- The first one is one dependence on the libraries on my maven application (javax.ws.rs.jar)

- The second one is a jar located in the directory with all the hadoop depdencies from hortonworks that I need to export into my classpath to make the application work (jsr311-api-1.1.1.jar)

 

So I understand  (but maybe I am wrokng) that I need to tell the program in some way that the maven dependency packaged in the jar of my application should be used instead of the jar located on the classpath.(javax.ws.rs.jar) without removing the jar in the classpath (obviously, it's part from my HDP cluster)

 

Any thoughts?

 

pom.xml dependency causing the issue:

 

 

<dependency>
<groupId>com.hortonworks.registries</groupId>
<artifactId>schema-registry-client</artifactId>
<version>${schema.registry.version}</version>
</dependency>

 

 

Dependency tree:

 

 

[INFO] amgrd:KafkaDemoProcessor:jar:0.0.1-SNAPSHOT
[INFO] +- org.apache.flink:flink-java:jar:1.8.1:compile
[INFO] | +- org.apache.flink:flink-core:jar:1.8.1:compile
[INFO] | | +- org.apache.flink:flink-annotations:jar:1.8.1:compile
[INFO] | | +- org.apache.flink:flink-metrics-core:jar:1.8.1:compile
[INFO] | | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile
[INFO] | | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | | \- org.objenesis:objenesis:jar:2.1:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | | \- org.apache.commons:commons-compress:jar:1.18:compile
[INFO] | +- org.apache.flink:flink-shaded-asm:jar:5.0.4-6.0:compile
[INFO] | +- org.apache.flink:flink-shaded-asm-6:jar:6.2.1-6.0:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.5:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.15:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | \- org.apache.flink:force-shading:jar:1.8.1:compile
[INFO] +- org.apache.flink:flink-streaming-java_2.12:jar:1.8.1:compile
[INFO] | +- org.apache.flink:flink-runtime_2.12:jar:1.8.1:compile
[INFO] | | +- org.apache.flink:flink-queryable-state-client-java_2.12:jar:1.8.1:compile
[INFO] | | +- org.apache.flink:flink-hadoop-fs:jar:1.8.1:compile
[INFO] | | +- org.apache.flink:flink-shaded-netty:jar:4.1.32.Final-6.0:compile
[INFO] | | +- org.apache.flink:flink-shaded-jackson:jar:2.7.9-6.0:compile
[INFO] | | +- org.javassist:javassist:jar:3.19.0-GA:compile
[INFO] | | +- org.scala-lang:scala-library:jar:2.12.7:compile
[INFO] | | +- com.typesafe.akka:akka-actor_2.12:jar:2.4.20:compile
[INFO] | | | +- com.typesafe:config:jar:1.3.0:compile
[INFO] | | | \- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.8.0:compile
[INFO] | | +- com.typesafe.akka:akka-stream_2.12:jar:2.4.20:compile
[INFO] | | | +- org.reactivestreams:reactive-streams:jar:1.0.0:compile
[INFO] | | | \- com.typesafe:ssl-config-core_2.12:jar:0.2.1:compile
[INFO] | | | \- org.scala-lang.modules:scala-parser-combinators_2.12:jar:1.0.4:compile
[INFO] | | +- com.typesafe.akka:akka-protobuf_2.12:jar:2.4.20:compile
[INFO] | | +- com.typesafe.akka:akka-slf4j_2.12:jar:2.4.20:compile
[INFO] | | +- org.clapper:grizzled-slf4j_2.12:jar:1.3.2:compile
[INFO] | | +- com.github.scopt:scopt_2.12:jar:3.5.0:compile
[INFO] | | +- org.xerial.snappy:snappy-java:jar:1.1.4:compile
[INFO] | | \- com.twitter:chill_2.12:jar:0.7.6:compile
[INFO] | | \- com.twitter:chill-java:jar:0.7.6:compile
[INFO] | \- org.apache.flink:flink-shaded-guava:jar:18.0-6.0:compile
[INFO] +- org.apache.flink:flink-clients_2.12:jar:1.8.1:compile
[INFO] | +- org.apache.flink:flink-optimizer_2.12:jar:1.8.1:compile
[INFO] | \- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] +- org.apache.flink:flink-connector-kafka_2.11:jar:1.8.1:compile
[INFO] | +- org.apache.flink:flink-connector-kafka-base_2.11:jar:1.8.1:compile
[INFO] | \- org.apache.kafka:kafka-clients:jar:2.0.1:compile
[INFO] | \- org.lz4:lz4-java:jar:1.4.1:compile
[INFO] +- com.hortonworks.registries:schema-registry-client:jar:0.3.0:compile
[INFO] | +- com.hortonworks.registries:schema-registry-common:jar:0.3.0:compile
[INFO] | | +- com.google.guava:guava:jar:18.0:compile
[INFO] | | +- org.slf4j:log4j-over-slf4j:jar:1.7.12:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.3:compile
[INFO] | | \- com.fasterxml.jackson.core:jackson-databind:jar:2.7.3:compile
[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.1:compile
[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] | | +- org.glassfish.jersey.core:jersey-common:jar:2.22.1:compile
[INFO] | | | +- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] | | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.1:compile
[INFO] | | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b31:compile
[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b31:compile
[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b31:compile
[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b31:compile
[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b31:compile
[INFO] | +- org.glassfish.jersey.media:jersey-media-multipart:jar:2.22.1:compile
[INFO] | | \- org.jvnet.mimepull:mimepull:jar:1.9.6:compile
[INFO] | +- com.hortonworks.registries:registry-common:jar:0.3.0:compile
[INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.3:compile
[INFO] | | | \- org.yaml:snakeyaml:jar:1.15:compile
[INFO] | | +- com.github.fge:json-schema-validator:jar:2.2.6:compile
[INFO] | | | +- joda-time:joda-time:jar:2.3:compile
[INFO] | | | +- com.googlecode.libphonenumber:libphonenumber:jar:6.2:compile
[INFO] | | | +- com.github.fge:json-schema-core:jar:1.2.5:compile
[INFO] | | | | +- com.github.fge:uri-template:jar:0.9:compile
[INFO] | | | | | \- com.github.fge:msg-simple:jar:1.1:compile
[INFO] | | | | | \- com.github.fge:btf:jar:1.2:compile
[INFO] | | | | +- com.github.fge:jackson-coreutils:jar:1.8:compile
[INFO] | | | | \- org.mozilla:rhino:jar:1.7R4:compile
[INFO] | | | +- javax.mail:mailapi:jar:1.4.3:compile
[INFO] | | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | | \- net.sf.jopt-simple:jopt-simple:jar:4.6:compile
[INFO] | | +- org.apache.hadoop:hadoop-client:jar:2.7.1:compile
[INFO] | | | +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
[INFO] | | | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | | | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | | | +- commons-codec:commons-codec:jar:1.4:compile
[INFO] | | | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | | | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] | | | | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | | | | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | | | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | | | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | | | | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | | | | +- org.apache.hadoop:hadoop-auth:jar:2.7.1:compile
[INFO] | | | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] | | | | | \- org.apache.curator:curator-framework:jar:2.7.1:compile
[INFO] | | | | +- org.apache.curator:curator-client:jar:2.7.1:compile
[INFO] | | | | +- org.apache.curator:curator-recipes:jar:2.7.1:compile
[INFO] | | | | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] | | | | \- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] | | | | \- io.netty:netty:jar:3.7.0.Final:compile
[INFO] | | | +- org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile
[INFO] | | | | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | | | | +- io.netty:netty-all:jar:4.0.23.Final:compile
[INFO] | | | | +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] | | | | | \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] | | | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | | | \- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
[INFO] | | +- org.hibernate:hibernate-validator:jar:5.1.3.Final:compile
[INFO] | | | +- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] | | | +- org.jboss.logging:jboss-logging:jar:3.1.3.GA:compile
[INFO] | | | \- com.fasterxml:classmate:jar:1.0.0:compile
[INFO] | | \- org.springframework:spring-context:jar:4.3.6.RELEASE:compile
[INFO] | | +- org.springframework:spring-aop:jar:4.3.6.RELEASE:compile
[INFO] | | +- org.springframework:spring-beans:jar:4.3.6.RELEASE:compile
[INFO] | | +- org.springframework:spring-core:jar:4.3.6.RELEASE:compile
[INFO] | | \- org.springframework:spring-expression:jar:4.3.6.RELEASE:compile
[INFO] | \- com.hortonworks.registries:common-auth:jar:0.3.0:compile
[INFO] | \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] \- com.hortonworks.registries:schema-registry-serdes:jar:0.3.0:compile
[INFO] +- org.apache.avro:avro:jar:1.8.2:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
[INFO] | \- org.tukaani:xz:jar:1.5:compile
[INFO] +- commons-io:commons-io:jar:2.5:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.3:compile
[INFO] +- org.glassfish.jersey.media:jersey-media-json-jackson:jar:2.22.1:compile
[INFO] | \- org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.22.1:compile
[INFO] \- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.7.3:compile
[INFO] +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.7.3:compile
[INFO] \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.3:compile

 

Thank you

 

Don't have an account?
Coming from Hortonworks? Activate your account here