Hello experts, We are running nifi 1.25.0 version in Kubernetes, single pod in cluster mode. Memory limit for the pod is 40G and JVM configured for Nifi is 28G. Now we are facing OOM error once in a while. When we checked nifi GUI System diagnostic it shows only 3G usage (Heap + non-heap) But when we checked at pod level memory usage for nifi process it shows very high usage. What else could be consuming so much memroy? This pod has only nifi container running Thanks in advance, Mahendra ... View more

Hello Experts, I am using below end point to send update variable registry request - "/process-groups/<id>/variable-registry/update-requests" and check the status using below url in loop "/process-groups/<id>/variable-registry/update-requests/<variableRegistryUpdateId>". It works well for some time and all of a sudden it starts behaving strange. Status will not get change even after 200 times status api call with almost 2 mins wait total. If we see the status api response , it shows like below - where it is stuck at 'Stopping affected Processors' stage , but if we look at effected processors, in one case there is not effected processors to stop and in another case all processors are already in stopped state but still it is not at progressing. Any idea what could be the reason? Update status check response  Case 1 : { "request": { "requestId": "573b0dd9-6f14-4ca2-80d1-513d93451f22", "uri": "https://o9nifi-1:9091/nifi-api/process-groups/9e391108-646a-3c3c-a642-a4fbce1d9107/variable-registry/573b0dd9-6f14-4ca2-80d1-513d93451f22", "submissionTime": "04/24/2025 13:47:09.647 GMT", "lastUpdated": "04/24/2025 13:47:09.647 GMT", "complete": false, "percentCompleted": 0, "updateSteps": [ { "description": "Identifying components affected", "complete": true }, { "description": "Stopping affected Processors", "complete": false }, { "description": "Disabling affected Controller Services", "complete": false }, { "description": "Applying Updates", "complete": false }, { "description": "Re-Enabling affected Controller Services", "complete": false }, { "description": "Restarting affected Processors", "complete": false } ], "processGroupId": "9e391108-646a-3c3c-a642-a4fbce1d9107", "affectedComponents": [] } }   Case 2: { "request" : { "requestId" : "b5ed4d79-13d2-4dfb-b556-8409e1ddfea6", "uri" : "https://o9nifi-1:9091/nifi-api/process-groups/9e391108-646a-3c3c-a642-a4fbce1d9107/variable-registry/b5ed4d79-13d2-4dfb-b556-8409e1ddfea6", "submissionTime" : "04/24/2025 16:52:00.067 GMT", "lastUpdated" : "04/24/2025 16:52:00.067 GMT", "complete" : false, "percentCompleted" : 0, "updateSteps" : [ { "description" : "Identifying components affected", "complete" : true }, { "description" : "Stopping affected Processors", "complete" : false }, { "description" : "Disabling affected Controller Services", "complete" : false }, { "description" : "Applying Updates", "complete" : false }, { "description" : "Re-Enabling affected Controller Services", "complete" : false }, { "description" : "Restarting affected Processors", "complete" : false } ], "processGroupId" : "9e391108-646a-3c3c-a642-a4fbce1d9107", "affectedComponents" : [ { "revision" : { "clientId" : "685687f8-0196-1000-340d-d586d2c6d581", "version" : 10 }, "id" : "2b0e0d15-5e1c-341c-87b1-421a39a81200", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "8a5d122b-b08b-30aa-b794-a2884fd31d45", "id" : "2b0e0d15-5e1c-341c-87b1-421a39a81200", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "8a5d122b-b08b-30aa-b794-a2884fd31d45", "name" : "test1" }, "referenceType" : "PROCESSOR" }, { "revision" : { "clientId" : "685687f8-0196-1000-340d-d586d2c6d581", "version" : 5 }, "id" : "11479904-0add-3d9f-b489-1024c46275c3", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "7a9a49dc-b361-394b-93d2-6cfb83f6b85c", "id" : "11479904-0add-3d9f-b489-1024c46275c3", "referenceType" : "PROCESSOR", "name" : "RouteOnAttribute", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "7a9a49dc-b361-394b-93d2-6cfb83f6b85c", "name" : "CheckRequestStatus" }, "referenceType" : "PROCESSOR" }, { "revision" : { "version" : 0 }, "id" : "2b0e0d15-5e1c-341c-a8d0-58edec551d2d", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "8a5d122b-b08b-30aa-a844-848d88e69396", "id" : "2b0e0d15-5e1c-341c-a8d0-58edec551d2d", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "8a5d122b-b08b-30aa-a844-848d88e69396", "name" : "two1" }, "referenceType" : "PROCESSOR" }, { "revision" : { "version" : 0 }, "id" : "2b0e0d15-5e1c-341c-893a-07179a37b42c", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "8a5d122b-b08b-30aa-a4a8-c6402262c60e", "id" : "2b0e0d15-5e1c-341c-893a-07179a37b42c", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "8a5d122b-b08b-30aa-a4a8-c6402262c60e", "name" : "two22" }, "referenceType" : "PROCESSOR" }, { "revision" : { "version" : 0 }, "id" : "2b0e0d15-5e1c-341c-acab-14e6adfe64da", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "8a5d122b-b08b-30aa-b227-72500a7c62cc", "id" : "2b0e0d15-5e1c-341c-acab-14e6adfe64da", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "8a5d122b-b08b-30aa-b227-72500a7c62cc", "name" : "two33" }, "referenceType" : "PROCESSOR" }, { "revision" : { "clientId" : "685687f8-0196-1000-340d-d586d2c6d581", "version" : 5 }, "id" : "2c48b295-2093-3bd2-a45a-d50c6d95cb8b", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "7a9a49dc-b361-394b-93d2-6cfb83f6b85c", "id" : "2c48b295-2093-3bd2-a45a-d50c6d95cb8b", "referenceType" : "PROCESSOR", "name" : "ReplaceText", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "7a9a49dc-b361-394b-93d2-6cfb83f6b85c", "name" : "CheckRequestStatus" }, "referenceType" : "PROCESSOR" } ] } }   Thanks, Mahendra ... View more

Hello experts -  I am using nifi 1.25 in Kubernetes. I have flow where it updates variable registry of a processor group . Invoking below url for the same -  <baseNiFiUrl>/process-groups/${ProcessGroupId}/variable-registry/update-requests It works for some time and once in a while it starts hanging, where it is unable to update the variable registry at and status stuck at processor stopping. Even I tried running update variable registry after manually stopping the specific processor of processor group where updating variable is being referenced. Here is the below response I get for var registry update status, it does not change: { "request" : { "requestId" : "77422664-ad85-40d7-9464-4df7df1fe149", "uri" : "https://nifi-0:9091/nifi-api/process-groups/fec6c2c9-ee27-31ea-6354-851d74803023/variable-registry/77422664-ad85-40d7-9464-4df7df1fe149", "submissionTime" : "12/20/2024 09:43:32.941 UTC", "lastUpdated" : "12/20/2024 09:43:32.941 UTC", "complete" : false, "percentCompleted" : 0, "updateSteps" : [ { "description" : "Identifying components affected", "complete" : true }, { "description" : "Stopping affected Processors", "complete" : false }, { "description" : "Disabling affected Controller Services", "complete" : false }, { "description" : "Applying Updates", "complete" : false }, { "description" : "Re-Enabling affected Controller Services", "complete" : false }, { "description" : "Restarting affected Processors", "complete" : false } ], "processGroupId" : "fec6c2c9-ee27-31ea-6354-851d74803023", "affectedComponents" : [ { "revision" : { "clientId" : "e3716811-0193-1000-9573-b7e6afd1da8d", "version" : 27 }, "id" : "f22a307d-36ea-37ac-515e-677404e7790b", "permissions" : { "canRead" : true, "canWrite" : true }, "bulletins" : [ ], "component" : { "processGroupId" : "fec6c2c9-ee27-31ea-6354-851d74803023", "id" : "f22a307d-36ea-37ac-515e-677404e7790b", "referenceType" : "PROCESSOR", "name" : "O9QueueRouting", "state" : "STOPPED", "activeThreadCount" : 0 }, "processGroup" : { "id" : "fec6c2c9-ee27-31ea-6354-851d74803023", "name" : "priceinputgspeandivandmarketidtest" }, "referenceType" : "PROCESSOR" } ] } } This will work for some time if I restart the nifi and after some time it will be back to square. Any suggestion would be much appreciated Thanks Mahendra ... View more

Hello Experts, I was using  "ConsumeAzureEventHub" processor with nifi 1.16.3 and when I configure 'Storage Container Name' field to store consumer group state, processor was automatically creating the container (if not present) in the storage account when processor was started. But in Nifi 1.25, I am seeing a different behavior where it does not auto create the container on processor start, instead just it show container does not exist error. Is this is the expected behaviour in 1.25? if so what is the solution? should we separately create the container before hand and then use in processor?    Thanks, Mahendra ... View more

Hello Experts, We have 2 node nifi cluster running on k8 cluster. We want to distribute the incoming http request on specific port to be load balanced across both nodes equally (round robin), do we have anyway in kubernetes for this? Tried headless service & ClusterIP but did not work as expected. Is there any other way to achieve this without external load balancers like AWS ELB etc. Thanks Mahendra ... View more