Member since
09-12-2017
2
Posts
0
Kudos Received
0
Solutions
01-24-2018
07:57 AM
Hi Suku: I response some of your questions: a) Which Keytab you have used, whether CM generated keytab or user keytab generated by you? I used kafka.keytab b) Path of your jaas.conf and keytab for Kafka? Path of kafka.keytab in /etc/security/keytabs/ c) How Kafka Kerberos configuration parameters set? The following is the configuration of Kafka parameters and the the form to use the jaas parameter. Properties props = new Properties(); props.put("bootstrap.servers", "xxxx:9092,xxx:9092"); props.put("client.id", "client-id-coprocessor "); props.put("key.serializer", StringSerializer.class.getName()); props.put("value.serializer", StringSerializer.class.getName()); props.put("security.protocol", "SASL_PLAINTEXT"); props.put("sasl.kerberos.service.name", "kafka"); props.put("sasl.jaas.config", "com.sun.security.auth.module.Krb5LoginModule required \n" + "useKeyTab=true \n" + "storeKey=true \n" + "keyTab=\"/etc/security/keytabs/kafka.keytab\" \n" + "principal=\"kafka/nodo@REALM\";"); KafkaProducer producer = new KafkaProducerString>(props); Remember sometimes you will need reboot your hbase service for deploy your coprocessor. I hope I will help you. Florentino
... View more
09-12-2017
01:17 PM
hi:
I want to integrate kerberos with kafka in hbase coprocessor and I could not autenticate inside the application. I am getting this error:
Caused by: java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode. at org.apache.kafka.common.security.kerberos.Login.login(Login.java:289) at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:104) at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44) at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:55) ... 14 more
This is part of my coprocessor postput code:
SparkConf conf = new SparkConf().setAppName("Coprocessor").setMaster("local[1]"); JavaSparkContext sc = new JavaSparkContext(conf); sc.getConf().set("spark.yarn.principal","user@EXAMPLE.COM"); sc.getConf().set("spark.yarn.keytab", "/home/user/user.keytab"); sc.getConf().set("spark.yarn.credentials.file", "credential_file");
Properties props = new Properties(); props.put("bootstrap.servers", "server.com:9092"); props.put("client.id", "client-id-coprocessor"); props.put("key.serializer", StringSerializer.class.getName()); props.put("value.serializer", StringSerializer.class.getName()); props.put("security.protocol","SASL_PLAINTEXT"); props.put("sasl.kerberos.service.name", "kafka");
KafkaProducer<String, String> producer = new KafkaProducer<String, String>(props); ProducerRecord<String, String> message = new ProducerRecord<String, String>(KAFKA_TOPIC,"key", "this is a simple message"); producer.send(message); producer.close();
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Kafka
-
Kerberos