Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas Hook for Kerberized Storm warns JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider

Atlas Hook for Kerberized Storm warns JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider

Explorer

Hi Community,

I have setup a HDP 2.6.3 kerberized cluster, that comprises Atlas, Kafka, Storm.

Now that I have kerberized the cluster, when I push a topology (storm jar mytopology.jar f.q.d.n.MyTopology topologyname), the AtlasHook has a problem determining keytab or ticket-cache for KafkaClient.

5667 [main] WARN  o.a.a.h.AtlasHook - Error in determining keytab for KafkaClient-JAAS config
java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider

This is really just a warning - in Atlas, I see the topology has been updated after completion of the command.

I suspected that I had conflicting jars files in the classpath but it looks like it's not the case.

This is the generated client_jaas.conf file

StormClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useTicketCache=true
   renewTicket=true
   serviceName="nimbus";
}; KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useTicketCache=true
   renewTicket=true
   serviceName="kafka";
};

The full stack trace look like follows.

1322 [main] INFO  o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -7645321177559766236:-6292660308148401340
1827 [main] INFO  o.a.s.m.n.Login - successfully logged in.
1904 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : gvscc-gw.novalocal:6627
2806 [main] INFO  o.a.s.s.a.AuthUtils - Got AutoCreds []
2812 [main] INFO  o.a.s.m.n.Login - successfully logged in.
2849 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : gvscc-gw.novalocal:6627
2901 [main] INFO  o.a.s.StormSubmitter - Uploading dependencies - jars...
2902 [main] INFO  o.a.s.StormSubmitter - Uploading dependencies - artifacts...
2902 [main] INFO  o.a.s.StormSubmitter - Dependency Blob keys - jars : [] / artifacts : []
2948 [main] INFO  o.a.s.StormSubmitter - Uploading topology jar /tmp/d606bdd4f21411e7af34fa174e2f5dce.jar to assigned location: /hadoop/storm/nimbus/inbox/stormjar-0d7bf304-4c62-4454-864a-571b4ada6662.jar
Start uploading file '/tmp/d606bdd4f21411e7af34fa174e2f5dce.jar' to '/hadoop/storm/nimbus/inbox/stormjar-0d7bf304-4c62-4454-864a-571b4ada6662.jar' (592151 bytes)
[==================================================] 592151 / 592151
File '/tmp/d606bdd4f21411e7af34fa174e2f5dce.jar' uploaded to '/hadoop/storm/nimbus/inbox/stormjar-0d7bf304-4c62-4454-864a-571b4ada6662.jar' (592151 bytes)
3002 [main] INFO  o.a.s.StormSubmitter - Successfully uploaded topology jar to assigned location: /hadoop/storm/nimbus/inbox/stormjar-0d7bf304-4c62-4454-864a-571b4ada6662.jar
3002 [main] INFO  o.a.s.StormSubmitter - Submitting topology exclamate in distributed mode with conf {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-7645321177559766236:-6292660308148401340","topology.workers":3,"topology.debug":true}
3548 [main] INFO  o.a.s.StormSubmitter - Finished submitting topology: exclamate
3548 [main] INFO  o.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
4875 [main] INFO  o.a.a.ApplicationProperties - Looking for atlas-application.properties in classpath
4876 [main] INFO  o.a.a.ApplicationProperties - Loading atlas-application.properties from file:/etc/storm/2.6.3.0-235/0/atlas-application.properties
log4j:WARN No appenders could be found for logger (org.apache.atlas.ApplicationProperties).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
5667 [main] WARN  o.a.a.h.AtlasHook - Error in determining keytab for KafkaClient-JAAS config
java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2273) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:101) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:291) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:846) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:816) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.isLoginKeytabBased(UserGroupInformation.java:1376) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.atlas.hook.AtlasHook.isLoginKeytabBased(AtlasHook.java:232) [atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:80) [atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_72]
at java.lang.Class.forName(Class.java:348) [?:1.8.0_72]
at org.apache.atlas.storm.hook.StormAtlasHook.initialize(StormAtlasHook.java:78) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.storm.hook.StormAtlasHook.<init>(StormAtlasHook.java:46) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_72]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0_72]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0_72]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0_72]
at java.lang.Class.newInstance(Class.java:442) [?:1.8.0_72]
at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at com.vrsn.vscc.ExclamationTopology.main(ExclamationTopology.java:60) [d606bdd4f21411e7af34fa174e2f5dce.jar:?]
Caused by: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2267) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
... 25 more
5700 [main] WARN  o.a.a.h.AtlasHook - Error in determining ticket-cache for KafkaClient-JAAS config
java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2273) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:101) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:291) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:846) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:816) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.isLoginTicketBased(UserGroupInformation.java:1384) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.atlas.hook.AtlasHook.isLoginTicketBased(AtlasHook.java:244) [atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:81) [atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_72]
at java.lang.Class.forName(Class.java:348) [?:1.8.0_72]
at org.apache.atlas.storm.hook.StormAtlasHook.initialize(StormAtlasHook.java:78) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.storm.hook.StormAtlasHook.<init>(StormAtlasHook.java:46) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_72]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0_72]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0_72]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0_72]
at java.lang.Class.newInstance(Class.java:442) [?:1.8.0_72]
at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at com.vrsn.vscc.ExclamationTopology.main(ExclamationTopology.java:60) [d606bdd4f21411e7af34fa174e2f5dce.jar:?]
Caused by: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2267) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
... 25 more
5801 [main] WARN  o.a.a.h.AtlasHook - Error in determining current user
java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2273) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:101) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:291) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:846) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:816) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:689) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
at org.apache.atlas.hook.AtlasHook.getUser(AtlasHook.java:221) ~[atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.hook.AtlasHook.getUser(AtlasHook.java:190) ~[atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:92) [atlas-notification-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_72]
at java.lang.Class.forName(Class.java:348) [?:1.8.0_72]
at org.apache.atlas.storm.hook.StormAtlasHook.initialize(StormAtlasHook.java:78) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at org.apache.atlas.storm.hook.StormAtlasHook.<init>(StormAtlasHook.java:46) [storm-bridge-shim-0.8.0.2.6.3.0-235.jar:0.8.0.2.6.3.0-235]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_72]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0_72]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0_72]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0_72]
at java.lang.Class.newInstance(Class.java:442) [?:1.8.0_72]
at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:426) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at org.apache.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:407) [storm-core-1.1.0.2.6.3.0-235.jar:1.1.0.2.6.3.0-235]
at com.vrsn.vscc.ExclamationTopology.main(ExclamationTopology.java:60) [d606bdd4f21411e7af34fa174e2f5dce.jar:?]
Caused by: java.lang.RuntimeException: class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2267) ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
... 26 more
5802 [main] INFO  o.a.a.h.AtlasHook - Created Atlas Hook
5816 [main] INFO  o.a.s.m.n.Login - successfully logged in.
5849 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : gvscc-gw.novalocal:6627
6103 [main] INFO  o.a.s.StormSubmitter - Invoking the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
6103 [main] INFO  o.a.a.s.h.StormAtlasHook - Collecting metadata for a new storm topology: exclamate
6164 [main] INFO  o.a.a.h.AtlasHook - Adding entity for type: storm_topology
7080 [main] INFO  o.a.k.c.p.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [gvscc-wn2.novalocal:6667]
buffer.memory = 33554432
client.id =
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 0
retry.backoff.ms = 100
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = kafka
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXTSASL
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
7089 [main] INFO  o.a.k.c.p.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [gvscc-wn2.novalocal:6667]
buffer.memory = 33554432
client.id = producer-1
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 0
retry.backoff.ms = 100
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = kafka
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXTSASL
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
7117 [main] INFO  o.a.k.c.s.a.AbstractLogin - Successfully logged in.
7127 [kafka-kerberos-refresh-thread] INFO  o.a.k.c.s.k.KerberosLogin - TGT refresh thread started.
7138 [kafka-kerberos-refresh-thread] INFO  o.a.k.c.s.k.KerberosLogin - TGT valid starting at: 2018-01-05T12:35:18.000+0000
7139 [kafka-kerberos-refresh-thread] INFO  o.a.k.c.s.k.KerberosLogin - TGT expires: 2018-01-06T12:35:18.000+0000
7139 [kafka-kerberos-refresh-thread] INFO  o.a.k.c.s.k.KerberosLogin - TGT refresh sleeping until: 2018-01-06T08:05:31.527+0000
7141 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'key.deserializer' was supplied but isn't a known config.
7141 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'value.deserializer' was supplied but isn't a known config.
7142 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'hook.group.id' was supplied but isn't a known config.
7142 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'zookeeper.connection.timeout.ms' was supplied but isn't a known config.
7143 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'zookeeper.session.timeout.ms' was supplied but isn't a known config.
7143 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'enable.auto.commit' was supplied but isn't a known config.
7144 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'zookeeper.connect' was supplied but isn't a known config.
7144 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'zookeeper.sync.time.ms' was supplied but isn't a known config.
7144 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'session.timeout.ms' was supplied but isn't a known config.
7144 [main] WARN  o.a.k.c.p.ProducerConfig - The configuration 'auto.offset.reset' was supplied but isn't a known config.
7146 [main] INFO  o.a.k.c.u.AppInfoParser - Kafka version : 0.10.1.2.6.3.0-235
7146 [main] INFO  o.a.k.c.u.AppInfoParser - Kafka commitId : ba0af6800a08d2f8

This Atlas bug seems to be very similar

https://issues.apache.org/jira/browse/ATLAS-1741

Anyone else saw that issue before ?

Thanks for your support!

-Regis

2 REPLIES 2
Highlighted

Re: Atlas Hook for Kerberized Storm warns JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider

Expert Contributor

@regis piccand , You are right this is warning message and can be ignored. if you need to fix this you need to set hadoop user group configs in hadoop core-site.xml

Highlighted

Re: Atlas Hook for Kerberized Storm warns JniBasedUnixGroupsMappingWithFallback not org.apache.hadoop.security.GroupMappingServiceProvider

New Contributor

Did u find solution , i am also facing same issue.

the property is in place - HDFS-core-site.xml

hadoop.security.group.mapping.hdp_shell=org.apache.hadoop.security.ShellBasedUnixGroupsMapping

@hadoop user , @

Don't have an account?
Coming from Hortonworks? Activate your account here