NiFi: how to select specific Kerberos encryption type for different applications?

[#1]

Hello, community. 😃

I have an interesting case with NiFi (for me =P). We have:

1. HDFS kerberized service that configured to use arcfour-hmac-md5 (rc4-hmac) for all default_tkt_enctype, default_tgs_enctypes and permitted_enctypes (yeap, it's legacy and now I cannot change it)
2. Other kerberized applications, e.g. Kafka with disabled arcfour-hmac-md5 (allow_weak_crypto = false, permitted_enctypes excludes arcfour-hmac-md5)

And I have no idea how to configure NiFi to communicate with that applications on time and use exactly supported TGS enctype for each one.

If I set default_tgs_enctype in krb5.conf on NiFi to:

default_tgs_enctypes = arcfour-hmac-md5 aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192

HDFS with arcfour-hmac-md5 works fine, but Kafka (where arcfour-hmac-md5 is disabled) fails. And vise versa -- moving arcfour-hmac-md5 down of the default_tgs_enctypes list broke HDFS communications but Kafka starts to work.

In krb5.conf I can override it in [appdefaults] section for different applications to workaround, but I don't know how to use it within NiFi and it's processors.

Is it possible? 

[#2]

Hello @asand3r, 

Glad to see you on the community. 

Directly on NiFi you cannot specify the those encryptions per processor. 

What comes to my mind is to configure per realm user, this should work. 
In the krb5.conf you can tell specifically for each realm user, something like this: 

[appdefaults]
hdfs = {
default_tgs_enctypes = arcfour-hmac-md5 aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96
permitted_enctypes = arcfour-hmac-md5 aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96
}

This will target any application using a principal with 'hdfs' in its name. You may need to be more specific in some cases, for example, using the full principal name.

In your NiFi HDFS processors, you'll need to set the Kerberos Principal property to a value that matches the [appdefaults] section.

[#3]

Hey, @vafs.

You know, I've tried that, but seems it doesn't work with Java applications.

I've add next section to my krb5.conf, that uses by NiFi server (I set custom file with nifi.kerberos.krb5.file):

[appdefaults]
service.name@DOMAIN.NAME.ORG = {
    default_tkt_enctypes = arcfour-hmac-md5 ...
    default_tgs_enctypes = arcfour-hmac-md5 ...
    permitted_enctypes = arcfour-hmac-md5 ...
    allow_weak_crypto = true
}

After that I restarted NiFi service just in case and nothing happen. But, if I return there options to [libdefaults] all works fine again.

Also, I've grep JDK sources to determine is [appdefaults] sections parsed by Java Kerberos implementation and find nothing. I think, the solution with [appdefaults] just shouldn't work. =(

[#4]

Yes, you're right. 
Looks like Java Kerberos makes the applications to not always have an application name that we can use here. 

I was reading about other option that makes the processes to fallback from one to another enctype. 
But that will need to have "allow_weak_crypto = true" and as you mentioned that is not possible in your scenario. 

Not sure if what you need is possible somehow. 

[#5]

I guess, that my problem has not solution from NiFi side and we just need to correct HDFS settings to accept other encryption types in addition to arcfour-hmac-md5.