Member since
05-14-2020
1
Post
0
Kudos Received
0
Solutions
05-14-2020
08:25 PM
Hello Team, am able to produce message to topic, but getting below error while running spark-kafka consumer , using com.sun.security.auth.module.Krb5LoginModule , jaas & krb5.conf file. running code through eclipse , am able to consume message when i run in terminal to using /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --topic <tn> --from-beginning --bootstrap-server <> --consumer.config <path>/consumer.properties exact error message : ERROR org.apache.spark.executor.Executor - Exception in task 2.0 in stage 0.0 (TID 2) org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: spark-executor-<groupID> Env Used : compile(group: 'org.apache.spark', name: 'spark-streaming_2.11', version: '2.3.1') compile(group: 'org.apache.spark', name: 'spark-streaming-kafka-0-10_2.11', version: '2.3.1') compile(group: 'org.apache.spark' , name: 'spark-core_2.11', version: '2.3.2') scala version : 2.11.8 val kafkaParam = Map[String, Object]( ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG -> broker, ConsumerConfig.GROUP_ID_CONFIG -> groupId, ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG -> "org.apache.kafka.common.serialization.StringDeserializer", ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG -> "org.apache.kafka.common.serialization.StringDeserializer" ,CommonClientConfigs.SECURITY_PROTOCOL_CONFIG -> "SASL_PLAINTEXT" )
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Spark