Hello Experts, We have 2 node nifi cluster running on k8 cluster. We want to distribute the incoming http request on specific port to be load balanced across both nodes equally (round robin), do we have anyway in kubernetes for this? Tried headless service & ClusterIP but did not work as expected. Is there any other way to achieve this without external load balancers like AWS ELB etc. Thanks Mahendra ... View more

Hello Experts, I have a nifi kafka consumer (ConsumeKafka_2_6) where the kafka message body (value/flow file content) and message key (kafka.key in flow file attributes) both are avro serialized as per the Confluent kafka way of  serializing. When we use "ConvertRecord + AvroReader CS + ConfluentSchemaRegistry CS"   to convert message body (value/flow file content) , it works fine as it is deserilising the magic byte and schema id to the correct value.   But we are trying below to deserilise kafka.key as well the same as value. Bringing the FF attribute kafka key to content (using ReplaceText processor with ${kafka.key}) and use "onvert Record + Avro Reader + ConfluentSchemaRegistry" to deserialize, in this case nifi is resulting in a wrong schema id - 3567 instead of the correct schema id 3545. Is it happening because when Nifi reads kafka.key (originally byte array) and pushes to downstream in FF as FF attribute and which is string? Is there any other way I can fix this or any other right approach?   Thanks in advance! @bbende @MattWho @mattw  Mahendra ... View more

Hello Experts, We have couple of Eventhub consumers running in nifi 1.16.3 version. The output connection is configured with default backpressure - 10,000 messages and 1GB. But I see 'ConsumeAzureEventhub' is keep on consuming data even after crossing backpressure. What is the reason for this behavior and how to fix this? @mattw    Thanks Mahendra     Thanks Mahendra ... View more

Hello experts, I am facing an issue in one of the Nifi server where we have multiple consume eventhub flows. The flow file repository disc is getting full but content and provenance repos are not.  Have attached the screen shot of all repos usage and content of flowfile repo. journals folder is occupying very large amount of data. nifi.properties (related to flofile repo): nifi.flowfile.repository.always.sync=false nifi.flowfile.repository.checkpoint.interval=2 mins nifi.flowfile.repository.directory=/flowfile nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository nifi.flowfile.repository.partitions=256 nifi.flowfile.repository.retain.orphaned.flowfiles=true nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog     Can anyone help me understand what is the issue? how to resolve this? Thanks, Mahendra   ... View more