Schema registry Kerberos Authentication required

cherifimohameda — Tue, 04 Dec 2018 01:32:56 GMT

Hi i'm using spring boot to write an api that will send an avro object to Kafka (producer). For this i'm using a kerberized schema registry. here is my code :

System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
System.setProperty("java.security.krb5.conf", API_KAFKA_KRB5);
Configuration conf = HBaseConfiguration.create();
conf.set("hadoop.security.authentication", "kerberos");
conf.set("hbase.security.authentication", "kerberos");
conf.set("hbase.security.authorization", "true");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(PRINCIPALE,KEYTAB);
ugi.doAs(new PrivilegedExceptionAction<Void>() {
	public Void run() throws Exception {
		producer.prepare(); // create kafka producer
		KafkaCallback kafkaCallback = new KafkaCallback();
		producer.push(API_KAFKA_TOPIC, null , smallFileService, kafkaCallback);
		return null;
	}
});

public void prepare() {
    System.setProperty("java.security.krb5.conf", API_KAFKA_KRB5);
    System.setProperty("java.security.auth.login.config", API_KAFKA_JAAS);

    Properties props = new Properties();
    props.put("bootstrap.servers", API_KAFKA_BROKER_LIST);
    props.put("schema.registry.url", registry);
    props.put("security.protocol", API_KAFKA_PROTOCOL);
    props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
    props.put("value.serializer",KafkaAvroSerializer.class.getName());
    props.put("request.required.acks", API_KAFKA_ACKS);
    props.put("ssl.truststore.password", API_KAFKA_TRUSTSTORE_PASSWORD);
    props.put("ssl.truststore.location", API_KAFKA_TRUSTSTORE_LOCATION);

    this.producer = new KafkaProducer<String, GenericRecord>(props);
}

public void push(String topic, String key, SmallFileService value, KafkaCallback kafkaCallback) {
    try {
        GenericRecord record = buildRecord(value);
        producer.send(new ProducerRecord<>(topic, key, record), kafkaCallback);
    }catch (IOException ie) {
        log.error(ie.getMessage());
    }
}

but i'm getting this error :

16:57:02.354 [http-nio-11002-exec-1] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is javax.ws.rs.NotAuthorizedException: HTTP 401 Authentication required] with root cause
javax.ws.rs.NotAuthorizedException: HTTP 401 Authentication required
        at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1002)
        at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:816)
        at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:92)
        at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:700)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
        at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:696)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:448)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:349)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$15.run(SchemaRegistryClient.java:1079)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$15.run(SchemaRegistryClient.java:1076)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.postEntity(SchemaRegistryClient.java:1076)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.doRegisterSchemaMetadata(SchemaRegistryClient.java:415)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaMetadata(SchemaRegistryClient.java:398)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.registerSchemaMetadata(SchemaRegistryClient.java:390)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaVersion(SchemaRegistryClient.java:443)
        at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.addSchemaVersion(SchemaRegistryClient.java:431)
        at com.hortonworks.registries.schemaregistry.serde.AbstractSnapshotSerializer.serialize(AbstractSnapshotSerializer.java:56)
        at com.hortonworks.registries.schemaregistry.serdes.avro.kafka.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:137)
        at com.hortonworks.registries.schemaregistry.serdes.avro.kafka.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:147)
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:783)
        at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:760)
        at fr.edf.dsp.loop.service.kafka.ProducerAvro.lambda$push$0(ProducerAvro.java:95)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)

how can i fix this ?

Thanks.

Re: Schema registry Kerberos Authentication required

cherifimohameda — Tue, 04 Dec 2018 22:08:18 GMT

Hello,

It may help someone someday, i fixed the problem by adding this to my /etc/krb5.conf [libdefaults] :

 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 allow_weak_crypto = false
 clockskew = 300
 rdns = false
 udp_preference_limit = 1

question Schema registry Kerberos Authentication required in Archives of Support Questions (Read Only)

Schema registry Kerberos Authentication required

Re: Schema registry Kerberos Authentication required