how to use apache Nifi on kerberized HDP Cluster nodes

[#1]

I have 7 node Kerberized HDP cluster. I have installed apache Nifi on one of my HDP cluster nodes just for testing purpose. When I try to configure putHdfs processor, the following warning pops up:

I tried to set the Kerberos properties as follows:

In addition to this, I set nifi.kerberos.krb5.file=/etc/krb5.conf

in the nifi.properties file. What is the correct configuration(on the Nifi host or HDFS host) for the putHdfs processor to work properly in this case?

Do I need to create Kerberos principal and Kerberos keytab file for the Nifi? Which service's principal or keytab file am I required to provide for "Kerberos Principal" and "Kerberos Keytab" fields in the putHdfs processor configuration?(is it the Nifi's or the Hdfs?)

Thanks,

[#2 · Accepted solution]

Hi @Kibrom Gebrehiwot,

You need to set the

nifi.kerberos.krb5.file=/etc/krb5.conf

property so that NiFi is able to know the information about your realms/controllers and negotiate tickets. Once this property is set, NiFi must be restarted.

Then, in the PutHDFS processor, you need to use the principal and keytab of the user you want to use to push data into HDFS. It is recommended to use the principal/keytab of a user you created if you have one.

Hope this helps.

[#3]

Thank you very much @Pierre Villard Your are answer was really helpful.

[#4]

Hi @pvillard,
I have some doubts about what kind of keytab file to create. I don't know much about Kerberos and do not find much information about the kind of Keytab file I should use to connect my Nifi Cluster to a kerberized Cloudera Cluster. Can you help me?

[#5]

The directory being specified is pointing to the NiFi server and not to the Hadoop Server Directory, please help,

Directory: /home/my_name is what i am keeping @Pierre Villard