About asand3r

asand3r · ‎09-21-2025

I guess, that my problem has not solution from NiFi side and we just need to correct HDFS settings to accept other encryption types in addition to arcfour-hmac-md5.

asand3r · ‎09-18-2025

@MattWho wrote: Now if there are network issues, Kafka may do a rebalance and assign more partitions to fewer consumers. We have an issue with JVM stop-the-world because. We are still using Java 8.0 in this cluster, so periodically JVM freezes about 5-15 seconds to perform GC. May it be a cause of LoadBalance issue?

asand3r · ‎09-18-2025

Hey, @vafs. You know, I've tried that, but seems it doesn't work with Java applications. I've add next section to my krb5.conf, that uses by NiFi server (I set custom file with nifi.kerberos.krb5.file): [appdefaults] service.name@DOMAIN.NAME.ORG = { default_tkt_enctypes = arcfour-hmac-md5 ... default_tgs_enctypes = arcfour-hmac-md5 ... permitted_enctypes = arcfour-hmac-md5 ... allow_weak_crypto = true } After that I restarted NiFi service just in case and nothing happen. But, if I return there options to [libdefaults] all works fine again. Also, I've grep JDK sources to determine is [appdefaults] sections parsed by Java Kerberos implementation and find nothing. I think, the solution with [appdefaults] just shouldn't work. =(

asand3r · ‎09-15-2025

@MattWho We have 30 partitions for that topic and Concurrent tasks set to 5.

asand3r · ‎09-14-2025

Hello, community. 😃 I have an interesting case with NiFi (for me =P). We have: 1. HDFS kerberized service that configured to use arcfour-hmac-md5 (rc4-hmac) for all default_tkt_enctype, default_tgs_enctypes and permitted_enctypes (yeap, it's legacy and now I cannot change it) 2. Other kerberized applications, e.g. Kafka with disabled arcfour-hmac-md5 (allow_weak_crypto = false, permitted_enctypes excludes arcfour-hmac-md5) And I have no idea how to configure NiFi to communicate with that applications on time and use exactly supported TGS enctype for each one. If I set default_tgs_enctype in krb5.conf on NiFi to: default_tgs_enctypes = arcfour-hmac-md5 aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 HDFS with arcfour-hmac-md5 works fine, but Kafka (where arcfour-hmac-md5 is disabled) fails. And vise versa -- moving arcfour-hmac-md5 down of the default_tgs_enctypes list broke HDFS communications but Kafka starts to work. In krb5.conf I can override it in [appdefaults] section for different applications to workaround, but I don't know how to use it within NiFi and it's processors. Is it possible?

asand3r · ‎09-14-2025

Thank you for such detailed answer. It's very helpful.

asand3r · ‎08-18-2025

Thanks, @MattWho for your points to LoadBalance. The 3rd node really had network connection issues that time, so maybe it takes place. For now it works fine, so I cannot do test steps that you offer. But I don't fully get your point about LB after ConsumeKafka. If Load balance is enabled is queue between ConsumeKafka and EvaluateJsonPath I can see that data is distributes along all cluster nodes in Data Provenance (look at screenshot below) , but if I disable it, only one node is presents here: My configuration with RoundRobin LB is wrong?

asand3r · ‎08-12-2025

Hey, communiy! We have a NiFi Cluster 1.18.0 (still so, yeap, sorry) and next issue do my mind. Simple flow, where we read data from Kafka with ConsumeKafka Processor and process it with EvaluateJsonPath after. One time we can see that queue between processors are stuck with data and nothing happens while EvaluateJsonPath or whole canvas not restared (right click -> stop -> start): Connection settings: Back Pressure Threshold: 10 000; Size Threshold: 1 Gb LB Strategy: Round Robin Queue threshold exceeded only on 3rd cluster node: Additional configuration data: Maximum timer driven thread count: 400 112 cores per node (2 CPU, 28 physical cores, 112 total with hyperthreading) So, as soon as I stop and start canvas -- all works fine again. Why it happens? How can I find the reason?

asand3r · ‎07-31-2025

OK, @MattWho, thanks for your help.

asand3r · ‎07-29-2025

But there methods requires authentication. As I know it's not possible to create internal NiFi service user with persistent auth token to make a health check with it. So, right way is HAProxy agent check?

Online	Offline
Last Visited	‎11-21-2025 01:44 AM

Member Since	‎05-14-2025 04:43 AM
Last Visited	‎11-21-2025 01:44 AM
Posts	24

Cloudera Community

Re: NiFi: how to select specific Kerberos encrypti...

Re: NiFi stuck data in queue between processors

Re: NiFi: how to select specific Kerberos encrypti...

Re: NiFi stuck data in queue between processors

NiFi: how to select specific Kerberos encryption t...

Re: NiFi stuck data in queue between processors

Re: NiFi stuck data in queue between processors

NiFi stuck data in queue between processors

Re: NiFi UI load balance with HAProxy to CONNECTED...

Re: NiFi UI load balance with HAProxy to CONNECTED...