Hello experts -  I am using nifi 1.25 in Kubernetes. I have flow where it updates variable registry of a processor group . Invoking below url for the same -  <baseNiFiUrl>/process-groups/${ProcessGroupId}/variable-registry/update-requests It works for some time and once in a while it starts hanging, where it is unable to update the variable registry at and status stuck at processor stopping. Even I tried running update variable registry after manually stopping the specific processor of processor group where updating variable is being referenced. Here is the below response I get for var registry update status, it does not change: { "request" : { "requestId" : "77422664-ad85-40d7-9464-4df7df1fe149", "uri" : "https://nifi-0:9091/nifi-api/process-groups/fec6c2c9-ee27-31ea-6354-851d74803023/variable-registry/77422664-ad85-40d7-9464-4df7df1fe149", "submissionTime" : "12/20/2024 09:43:32.941 UTC", "lastUpdated" : "12/20/2024 09:43:32.941 UTC", "complete" : false, "percentCompleted" : 0, "updateSteps" : [ { "description" : "Identifying components affected", "complete" : true }, { "description" : "Stopping affected Processors", "complete" : false }, { "description" : "Disabling affected Controller Services", "complete" : false }, { "description" : "Applying Updates", "complete" : false }, { "description" : "Re-Enabling affected Controller Services", "complete" : false }, { "description" : "Restarting affected Processors", "complete" : false } ], "processGroupId" : "fec6c2c9-ee27-31ea-6354-851d74803023", "affectedComponents" : [ { "revision" : { "clientId" : "e3716811-0193-1000-9573-b7e6afd1da8d", "version" : 27 }, "id" : "f22a307d-36ea-37ac-515e-677404e7790b", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "fec6c2c9-ee27-31ea-6354-851d74803023", "id" : "f22a307d-36ea-37ac-515e-677404e7790b", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "fec6c2c9-ee27-31ea-6354-851d74803023", "name" : "priceinputgspeandivandmarketidtest" }, "referenceType" : "PROCESSOR" } ] } } This will work for some time if I restart the nifi and after some time it will be back to square. Any suggestion would be much appreciated Thanks Mahendra ... View more

Hello Experts, I was using  "ConsumeAzureEventHub" processor with nifi 1.16.3 and when I configure 'Storage Container Name' field to store consumer group state, processor was automatically creating the container (if not present) in the storage account when processor was started. But in Nifi 1.25, I am seeing a different behavior where it does not auto create the container on processor start, instead just it show container does not exist error. Is this is the expected behaviour in 1.25? if so what is the solution? should we separately create the container before hand and then use in processor?    Thanks, Mahendra ... View more

Hello Experts, We have 2 node nifi cluster running on k8 cluster. We want to distribute the incoming http request on specific port to be load balanced across both nodes equally (round robin), do we have anyway in kubernetes for this? Tried headless service & ClusterIP but did not work as expected. Is there any other way to achieve this without external load balancers like AWS ELB etc. Thanks Mahendra ... View more

Hello Experts, I have a nifi kafka consumer (ConsumeKafka_2_6) where the kafka message body (value/flow file content) and message key (kafka.key in flow file attributes) both are avro serialized as per the Confluent kafka way of  serializing. When we use "ConvertRecord + AvroReader CS + ConfluentSchemaRegistry CS"   to convert message body (value/flow file content) , it works fine as it is deserilising the magic byte and schema id to the correct value.   But we are trying below to deserilise kafka.key as well the same as value. Bringing the FF attribute kafka key to content (using ReplaceText processor with ${kafka.key}) and use "onvert Record + Avro Reader + ConfluentSchemaRegistry" to deserialize, in this case nifi is resulting in a wrong schema id - 3567 instead of the correct schema id 3545. Is it happening because when Nifi reads kafka.key (originally byte array) and pushes to downstream in FF as FF attribute and which is string? Is there any other way I can fix this or any other right approach?   Thanks in advance! @bbende @MattWho @mattw  Mahendra ... View more

Hello Experts, We have couple of Eventhub consumers running in nifi 1.16.3 version. The output connection is configured with default backpressure - 10,000 messages and 1GB. But I see 'ConsumeAzureEventhub' is keep on consuming data even after crossing backpressure. What is the reason for this behavior and how to fix this? @mattw    Thanks Mahendra     Thanks Mahendra ... View more