JOLT Guide for Apache NiFi

[#1]

Hi everyone, I hope you’re doing well. I am working on a dataflow in Apache NiFi 1.18, and I need to retrieve the queue size information (flowfile count and content size) directly within NiFi itself, not via an external script or Postman. I know that the NiFi REST API provides this data, and I can access it successfully using external tools. However, my goal is to access queue metrics from inside NiFi, for example through processors like InvokeScriptedProcessor, QueryNiFiReportingTask, or any other built-in mechanism, without sending an external REST API request from outside NiFi. Is there a recommended approach, processor, or reporting task that allows NiFi to read its own queue sizes internally? If not, what would be the best practice to achieve this? Any guidance or examples would be greatly appreciated. Thank you in advance!

[#2]

Hello @hckorkmaz01, 

I was testing one way that in CFM didn't worked, but in your case, in NiFi directly, it may work, you can test. 

Under NiFi settings, create a reporting task like this: 

You will need to use this QueryNiFiReportingTask. 
This query should work for what you need: 

SELECT
QUEUE_ID,
QUEUE_NAME,
FLOWFILE_COUNT,
CONTENT_SIZE
FROM FLOWFILE_QUEUE_STATS

Then save the output to either log or any other method you prefer. 

Another option I was thinking is to use InvokeHTTP processor. 
To get the information you need. 
That will be in NiFi but using the REST API: 

GET http://localhost:8080/nifi-api/flow/process-groups/<PG_ID>/connections

That could be an option for you. 

Maybe test this and let us know if that worked. 

[#3]

@hckorkmaz01 

@vafs provided a couple excellent options already.

As a third option, you could utilize the SiteToSiteStatusReportingTask.  You can configure this NiFi reporting task to send metrics for only connections and you can add a regex to limited the number of connections returned (this requires renaming the desired connections so you have more granular control with the regex).  This Reporting task can be configured to send to a remote input port on on the same NiFi.  

This Reporting task will execute the query on each host in your NiFi Cluster and send results to the the desired Site-To-Site remote input port.   So in a three node NiFi cluster three FlowFiles will be produced and sent with Json content for connection(s) you filtered on.   This way you can see what is queued per node. 

Limitations to monitoring connections in this way exist.  Keep in mind that you are grabbing a snapshot at one specific moment in time.  So if the downstream component of this monitored connection is running, the connection stat would have likely changed by the time you got the last status details.   Now if you are using this to monitor flow dead-ends where you don't expect FlowFiles to ever accumulate, that is perhaps a valid use case.

Example json output placed in FlowFile:

[ {
  "statusId" : "27b01368-8280-4e5e-ac22-0e749c46fe17",
  "timestampMillis" : 1771336855410,
  "timestamp" : "2026-02-17T14:00:55.410Z",
  "actorHostname" : "nifi-node-1",
  "componentType" : "Connection",
  "componentName" : "Monitored-success",
  "parentId" : "603017c1-0197-1000-0000-000013171e7c",
  "parentName" : "test",
  "parentPath" : "NiFi Flow / test",
  "platform" : "nifi",
  "application" : "NiFi Flow",
  "componentId" : "508a2293-019a-1000-ffff-fffff15dc582",
  "sourceId" : "5089d4ab-019a-1000-ffff-ffff9b682ab3",
  "sourceName" : "UpdateAttribute",
  "destinationId" : "4dc23b89-25b0-1eff-b974-6fe5425c283f",
  "destinationName" : "UpdateAttribute",
  "maxQueuedBytes" : 0,
  "maxQueuedCount" : 0,
  "queuedBytes" : 30720,
  "queuedCount" : 30,
  "inputBytes" : 0,
  "inputCount" : 0,
  "outputBytes" : 0,
  "outputCount" : 0,
  "backPressureBytesThreshold" : 1073741824,
  "backPressureObjectThreshold" : 10000,
  "backPressureDataSizeThreshold" : "1 GB",
  "isBackPressureEnabled" : "false"
} ]

IMPORTANT NOTE:

I'd also like to point out that Apache NiFi 1.18 was released back in 2021 and newer releases have addressed many bugs and CVEs.  I strongly encourage you to transition to a newer version of Apache NiFi.  If you aren't ready to migrate to the newer Apache NiFi 2.x major release versions, you can still easily upgrade to last Apache NiFi 1.x major release version (1.28.0).

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt