Created 09-29-2017 07:15 PM
I am getting a login failure when trying to use the PutHDFS processor.
These are the things I have verified:
Environment is HDF 3.0.1 (NiFi 1.2) to HDP 2.6.x
Is there something I am missing?
Created 09-29-2017 07:26 PM
Did you copy over the core-site.xml and hdfs-site.xml files from the HDP cluster after enabling kerberos?
Created 09-29-2017 07:28 PM
Yes, and verified permissions on those files as well.
Created 09-29-2017 07:53 PM
How is the PutHDFS processor configured?
Created on 09-29-2017 08:19 PM - edited 08-17-2019 11:35 PM
Other than the config xmls and kerberos everything is default.
Created 09-29-2017 08:41 PM
Is this a cluster or standalone instance?
Also, would you post to complete error stack from the nifi-app.log file?
Created 09-29-2017 08:49 PM
Both HDP and HDF are clustered.
2017-09-29 16:42:41,239 ERROR [StandardProcessScheduler Thread-6] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480) at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102) at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1301) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1463) ... 9 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1305) at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1301) ... 6 common frames omitted Caused by: java.io.IOException: Login failure for NiFi_App_Prod from keytab /etc/security/keytabs/NiFi_App_Prod.keytab at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1183) at org.apache.nifi.hadoop.SecurityUtil.loginKerberos(SecurityUtil.java:54) at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:271) at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:204) at org.apache.nifi.processors.hadoop.PutHDFS.onScheduled(PutHDFS.java:191) ... 16 common frames omitted Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) at sun.reflect.GeneratedMethodAccessor609.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1172) ... 20 common frames omitted
Created 09-29-2017 09:00 PM
You need to use a headless keytab, so it does not require a password. There is no way to provide the password in the processor.
Created 09-29-2017 09:14 PM
The existing keytab should not require a password. The following command works, and doesn't prompt for a password.
kinit -kt /etc/security/keytabs/NiFi_App_Prod.keytab NiFi_App_Prod@<REALM>.com
Also, a similar configuration works in the dev environment without issue.
Created 09-30-2017 08:36 AM
Can you try to validate by doing the below steps
$ kdestroy
The grab a ticket as the nifi user
[root@host ~]# sudo su - nifi_user $ kinit Password for nifi_user@NAM.xxxxxxxx.COM: $ klist Ticket cache: FILE:/tmp/krb5cc_49393 Default principal: nifi_user@NAM.xxxxxxxx.COM Valid starting Expires Service principal 09/28/17 17:06:44 09/29/17 03:06:44 krbtgt/ NAM.xxxxxxxx.COM@NAM.xxxxxxxx.COM
Tell me if that works