Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

MiNiFi - How to get processors list and number of queued flowfiles?

Highlighted

MiNiFi - How to get processors list and number of queued flowfiles?

New Contributor

I'd like to monitor state of the running MiNiFi flow, especially get list of the processors and number of queued flowfiles for each processor. I'm trying to use FlowStatus Script Query, eg.:

$ ./minifi.sh flowStatus systemdiagnostics:processorstats
{"controllerServiceStatusList":null,"processorStatusList":null,"connectionStatusList":null,"remoteProcessGroupStatusList":null,"instanceStatus":null,"systemDiagnosticsStatus":{"garbageCollectionStatusList":null,"heapStatus":null,"contentRepositoryUsageList":null,"flowfileRepositoryUsage":null,"processorStatus":{"loadAverage":1.99,"availableProcessors":2}},"reportingTaskStatusList":null,"errorsGeneratingReport":[]}
 $ ./minifi.sh flowStatus processor:all:health,stats,bulletins
{"controllerServiceStatusList":null,"processorStatusList":[],"connectionStatusList":null,"remoteProcessGroupStatusList":null,"instanceStatus":null,"systemDiagnosticsStatus":null,"reportingTaskStatusList":null,"errorsGeneratingReport":[]}
 $ /minifi.sh flowStatus processor:MyProcessorName:health,stats,bulletins
{"controllerServiceStatusList":null,"processorStatusList":[],"connectionStatusList":null,"remoteProcessGroupStatusList":null,"instanceStatus":null,"systemDiagnosticsStatus":null,"reportingTaskStatusList":null,"errorsGeneratingReport":["Unable to get status for request 'processor:MyProcessorName:health,stats,bulletins' due to:org.apache.nifi.minifi.status.StatusRequestException: No processor with key MyProcessorName to report status on"]}

but I'm receiving only nulls (although my config.yml contains one Process Group and several processors as its children. The flow is running and working - I can receive correct HTTP response from HandleHttpResponse processor). What should I do to be able retrieve data which I want (enable some option in config?)? Is it possible using flowStatus queries? My flow which is running contains several processors, so why systemdiagnostics shows only two availableProcessors and why I can't use flowStatus processor command to get any processor data?

I didn't find any detailed information in the (Mi)NiFi documentation, so I'm not even sure if I can retrieve processors data (number of queued elements and processors list) in this way. If not, maybe do you know some other way to do it?

Please also tell me if am I right that MiNiFi doesn't provide REST API like NiFi (nifi.apache.org/docs/nifi-docs/rest-api/index.html)? Querying interface is the only way of getting information about the flow?

3 REPLIES 3

Re: MiNiFi - How to get processors list and number of queued flowfiles?

New Contributor

Any ideas? What is the best and working way of monitoring MiNiFi flow status?

Re: MiNiFi - How to get processors list and number of queued flowfiles?

New Contributor

Any ideas? I am facing the same issue and when i query for an specific connection/processor I always get :

"Unable to get status for request 'processor:05457aa4-55f2-35d4-0000-000000000000:health,stats,bulletins' due to:org.apache.nifi.minifi.status.StatusRequestException: No processor with key .... to report status on"]

Re: MiNiFi - How to get processors list and number of queued flowfiles?

New Contributor

Did you solve this problem? 

Don't have an account?
Coming from Hortonworks? Activate your account here