Support Questions

Find answers, ask questions, and share your expertise

Schema registry Kerberos Authentication required

New Contributor

Hi i'm using spring boot to write an api that will send an avro object to Kafka (producer). For this i'm using a kerberized schema registry. here is my code :

System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
System.setProperty("java.security.krb5.conf", API_KAFKA_KRB5);
Configuration conf = HBaseConfiguration.create();
conf.set("hadoop.security.authentication", "kerberos");
conf.set("hbase.security.authentication", "kerberos");
conf.set("hbase.security.authorization", "true");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(PRINCIPALE,KEYTAB);
ugi.doAs(new PrivilegedExceptionAction<Void>() {
	public Void run() throws Exception {
		producer.prepare(); // create kafka producer
		KafkaCallback kafkaCallback = new KafkaCallback();
		producer.push(API_KAFKA_TOPIC, null , smallFileService, kafkaCallback);
		return null;
	}
});
public void prepare() {
    System.setProperty("java.security.krb5.conf", API_KAFKA_KRB5);
    System.setProperty("java.security.auth.login.config", API_KAFKA_JAAS);

    Properties props = new Properties();
    props.put("bootstrap.servers", API_KAFKA_BROKER_LIST);
    props.put("schema.registry.url", registry);
    props.put("security.protocol", API_KAFKA_PROTOCOL);
    props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
    props.put("value.serializer",KafkaAvroSerializer.class.getName());
    props.put("request.required.acks", API_KAFKA_ACKS);
    props.put("ssl.truststore.password", API_KAFKA_TRUSTSTORE_PASSWORD);
    props.put("ssl.truststore.location", API_KAFKA_TRUSTSTORE_LOCATION);

    this.producer = new KafkaProducer<String, GenericRecord>(props);
}
public void push(String topic, String key, SmallFileService value, KafkaCallback kafkaCallback) {
    try {
        GenericRecord record = buildRecord(value);
        producer.send(new ProducerRecord<>(topic, key, record), kafkaCallback);
    }catch (IOException ie) {
        log.error(ie.getMessage());
    }
}

but i'm getting this error :

16:57:02.354 [http-nio-11002-exec-1] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is javax.ws.rs.NotAuthorizedException: HTTP 401 Authentication required] with root cause
javax.ws.rs.NotAuthorizedException: HTTP 401 Authentication required
        at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1002)
        at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:816)
        at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:92)
        at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:700)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
        at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:696)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:448)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:349)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$15.run(SchemaRegistryClient.java:1079)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$15.run(SchemaRegistryClient.java:1076)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.postEntity(SchemaRegistryClient.java:1076)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.doRegisterSchemaMetadata(SchemaRegistryClient.java:415)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaMetadata(SchemaRegistryClient.java:398)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.registerSchemaMetadata(SchemaRegistryClient.java:390)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaVersion(SchemaRegistryClient.java:443)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaVersion(SchemaRegistryClient.java:431)
        at com.hortonworks.registries.schemaregistry.serde.AbstractSnapshotSerializer.serialize(AbstractSnapshotSerializer.java:56)
        at com.hortonworks.registries.schemaregistry.serdes.avro.kafka.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:137)
        at com.hortonworks.registries.schemaregistry.serdes.avro.kafka.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:147)
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:783)
        at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:760)
        at fr.edf.dsp.loop.service.kafka.ProducerAvro.lambda$push$0(ProducerAvro.java:95)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)

how can i fix this ?

Thanks.

1 ACCEPTED SOLUTION

New Contributor

Hello,

It may help someone someday, i fixed the problem by adding this to my /etc/krb5.conf [libdefaults] :

 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 allow_weak_crypto = false
 clockskew = 300
 rdns = false
 udp_preference_limit = 1

View solution in original post

1 REPLY 1

New Contributor

Hello,

It may help someone someday, i fixed the problem by adding this to my /etc/krb5.conf [libdefaults] :

 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 allow_weak_crypto = false
 clockskew = 300
 rdns = false
 udp_preference_limit = 1
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.