Support Questions
Find answers, ask questions, and share your expertise

[NIFI PROCESSOR] : monitor : data send or recieve by processor

Explorer

hi all,

I'm trying use API REST NIFI but i don't find how can i extract the among of data send or recieved by a processor like listenHTTP, putHDFS, putSFTP.

thanks for helps.

3 REPLIES 3

Re: [NIFI PROCESSOR] : monitor : data send or recieve by processor

@mayki wogno

The Status History of a processor, which is seen in the UI by right clicking on a processor, then select Status History

14956-screen-shot-2017-05-01-at-104841-am.png

Another window pops-up, that looks like

14958-screen-shot-2017-05-01-at-105037-am.png

There are multiple statistics that can be viewed.

You can get the Status History of a processor via the rest api by using a command similar to this:

curl 'http://{nifi-host}:{port}/nifi-api/flow/processors/{id}/status/history'

You can get the same information for connection, buy replacing processors with connections. It will return the complete history available based on the configuration of NiFi. By default it will have 24 hours of status history.

You can get that information via the processor call, for example, this is the command I used to get the information, for the last 5 minutes, from a MonitorActivity processor:

curl 'http://{nifi-host}:{port}/nifi-api/processors/{id}'

This command will return a complete dump of the processor information, at the bottom of the information returned you will see:

"bytesRead":0,"bytesWritten":0,"read":"0 bytes","written":"0 bytes","flowFilesIn":299,"bytesIn":299,"input":"299 (299 bytes)","flowFilesOut":299,"bytesOut":299,"output":"299 (299 bytes)","taskCount":30,"tasksDurationNanos":52638168,"tasks":"30","tasksDuration":"00:00:00.052"

Re: [NIFI PROCESSOR] : monitor : data send or recieve by processor

Master Guru

@mayki wogno

I not sure I am clear on your question.

Are yo trying to use the rest api to get a listing of all files received by ListenHTTP and send using putHDFS? If so that would need to be done using Provenance.

So first you need the UUID for the processor you are going to running the provence query against

Lets assume your are looking for all Received Files between x and y timeframe. You would do the following commands:

### produce query running the following command:

curl 'http://<nifi hostname>:<port>/nifi-api/provenance' -H 'Content-Type: application/json' --data-binary '{"provenance":{"request":{"maxResults":1000,"startDate":"04/28/2017 18:35:00 UTC","endDate":"04/28/2017 23:59:59 UTC","searchTerms":{"EventType":"RECIEVE","ProcessorID":"<UUID of processor>"}}}}' --compressed

### From the response get the provenance id. You will need it in the next command to get the results:

curl 'http://<nifi hostname>:<port>/nifi-api/provenance/<provenance id>' --compressed

It will be returned on JSON format.

For your putHDFS processor you would be looking for "SEND" events.

Thanks,

Matt

Re: [NIFI PROCESSOR] : monitor : data send or recieve by processor

Explorer

@Matt Clarke and @Wynner : thanks i will check this information