standalone NiFi + putKafka into kerberized Kafka throws error "java.io.IOException: Configuration Error: Line 7: expected [option key]"

[#1]

Hi,

I want to use standalone NiFi 0.6 to ingest data into Kafka within a kerberized HDP2.3.4 cluster.

Base config ran smooth and additional config from https://community.hortonworks.com/articles/28180/how-to-configure-hdf-12-to-send-to-and-get-data-fr.... , "Section 2" has been applied.

To start playing around I added a GetFile processor and connected it to PutKafka, but after starting the flow, PutKafka fails with the following error:

2016-09-20 11:44:38,076 INFO [StandardProcessScheduler Thread-6] o.a.kafka.clients.producer.KafkaProducer Closing the Kafka producer with timeoutMillis = 0 ms.
2016-09-20 11:44:38,076 ERROR [StandardProcessScheduler Thread-6] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=9293cf22-d7c2-40a6-ba7e-a6d41942fc30] PutKafka[id=9293cf22-d7c2-40a6-ba7e-a6d41942fc30] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30000 milliseconds: java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.
2016-09-20 11:44:38,082 ERROR [StandardProcessScheduler Thread-6] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally
java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.7.0_79]
        at java.util.concurrent.FutureTask.get(FutureTask.java:188) ~[na:1.7.0_79]
        at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) ~[na:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: java.lang.reflect.InvocationTargetException: null
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_79]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_79]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na]
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na]
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na]
        at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1247) ~[na:na]
        at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1243) ~[na:na]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_79]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) ~[na:1.7.0_79]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) ~[na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
        ... 2 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka producer
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:275) ~[na:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:176) ~[na:na]
        at org.apache.nifi.processors.kafka.KafkaPublisher.<init>(KafkaPublisher.java:66) ~[na:na]
        at org.apache.nifi.processors.kafka.PutKafka.createKafkaPublisher(PutKafka.java:281) ~[na:na]
        ... 15 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
        at org.apache.kafka.common.network.Selector.<init>(Selector.java:135) ~[na:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:231) ~[na:na]
        ... 18 common frames omitted
Caused by: java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
        at org.apache.kafka.common.security.kerberos.Login.login(Login.java:306) ~[na:na]
        at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:101) ~[na:na]
        at org.apache.kafka.common.security.kerberos.LoginFactory.<init>(LoginFactory.java:34) ~[na:na]
        at org.apache.kafka.common.network.Selector.<init>(Selector.java:132) ~[na:na]
        ... 19 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
        at java.security.Provider$Service.newInstance(Provider.java:1259) ~[na:1.7.0_79]
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:243) ~[na:1.7.0_79]
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:190) ~[na:1.7.0_79]
        at javax.security.auth.login.Configuration.getInstance(Configuration.java:352) ~[na:1.7.0_79]
        at org.apache.kafka.common.security.kerberos.Login.login(Login.java:303) ~[na:na]
        ... 22 common frames omitted
Caused by: java.io.IOException: Configuration Error:
        Line 7: expected [option key]
        at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:550) ~[na:1.7.0_79]
        at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:439) ~[na:1.7.0_79]
        at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383) ~[na:1.7.0_79]
        at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283) ~[na:1.7.0_79]
        at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166) ~[na:1.7.0_79]
        at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124) ~[na:1.7.0_79]
        at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72) ~[na:1.7.0_79]
        at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61) ~[na:1.7.0_79]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_79]
        at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61) ~[na:1.7.0_79]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_79]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_79]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_79]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_79]
        at java.security.Provider$Service.newInstance(Provider.java:1254) ~[na:1.7.0_79]
        ... 26 common frames omitted

The PutKafka processor looks like:

and the zookeeper-jaas.conf looks like ("REALM" has been replaced):

#>cat /data/nifi/current/conf/zookeeper-jaas.conf
Client {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="/etc/security/keytabs/nifi.user.keytab"
  storeKey=true
  useTicketCache=false
  principal="nifi@REALM;
};
KafkaClient {
  com.sun.security.auth.module.Krb5LoginModule required
  useTicketCache=true
  renewTicket=true
  serviceName="kafka"
  useKeyTab=true
  keyTab="/etc/security/keytabs/nifi.user.keytab"
  principal="nifi@REALM";
};

=> keytab file is in place and readable

Any hints what is causing this "Configuration error" in the PutKafka processor ?

Thanks...

[#2 · Accepted solution]

@Gerd Koenig

After a closer look at the jaas file you posted above, I believe you issue is because of a missing " in the following line:

principal="nifi@REALM;

This line should actually be:

principal="nifi@REALM";

Try making the above change and restarting your NiFi.

Thanks,

Matt

[#3]

OMG, stupid me 😄

Thanks @mclark , exactly that solved the issue, sorry for bothering