Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Apache minifi Monitoring

Apache minifi Monitoring

Contributor

Hi all,

I am trying to understand the output of the minifi.sh flowfile <options> commands.

I wanna be able to monitor protectively all the "edge points" created by minifi.

Example:

The is the output of this cmd

/minifi.sh  flowStatus systemdiagnostics:heap,processorstats,contentrepositoryusage,flowfilerepositoryusage,garbagecollection

Output

{
	"controllerServiceStatusList": null,
	"processorStatusList": null,
	"connectionStatusList": null,
	"remoteProcessGroupStatusList": null,
	"instanceStatus": null,
	"systemDiagnosticsStatus": {
		"garbageCollectionStatusList": [{
			"name": null,
			"collectionCount": 3067,
			"collectionTime": 26764
		},
		{
			"name": null,
			"collectionCount": 6,
			"collectionTime": 861
		}],
		"heapStatus": {
			"totalHeap": 260571136,
			"maxHeap": 260571136,
			"freeHeap": 100187704,
			"usedHeap": 160383432,
			"heapUtilization": 62,
			"totalNonHeap": 76414976,
			"maxNonHeap": -1,
			"freeNonHeap": 2070192,
			"usedNonHeap": 74344784,
			"nonHeapUtilization": -1
		},
		"contentRepositoryUsageList": [{
			"name": "default",
			"freeSpace": 11783532544,
			"totalSpace": 29994987520,
			"usedSpace": 18211454976,
			"diskUtilization": 61
		}],
		"flowfileRepositoryUsage": {
			"freeSpace": 11783532544,
			"totalSpace": 29994987520,
			"usedSpace": 18211454976,
			"diskUtilization": 61
		},
		"processorStatus": {
			"loadAverage": 10.52,
			"availableProcessors": 14
		}
	},
	"reportingTaskStatusList": null,
	"errorsGeneratingReport": []
}

Some of the metrics listed are self explanatory but not in all cases and i would hate to make wrong assumptions and build a process around wrong interpreted metrics.

Where do i get stuck ?!

- trying to capture connection metrics/values

/minifi.sh flowStatus connection:all:health,stats

And i have Questions like :

connextionstatus.inputCount: 25 <--- this value is this since begging ? is this based on 5 min interval ? How to i interpret this ?

{
	"controllerServiceStatusList": null,
	"processorStatusList": null,
	"connectionStatusList": [{
		"name": "OUT Monitor - Clean and Extract//015f11b2-ba5c-14f0-d0dd-f17188d247d8",
		"connectionHealth": {
			"queuedCount": 0,
			"queuedBytes": 0
		},
		"connectionStats": {
			"inputCount": 25,
			"inputBytes": 3584,
			"outputCount": 25,
			"outputBytes": 3584
		}
	},
	{
		"name": "OUT Monitor - Initiate //IN Monitor - Clean and Extract",
		"connectionHealth": {
			"queuedCount": 0,
			"queuedBytes": 0
		},
		"connectionStats": {
			"inputCount": 6,
			"inputBytes": 828,
			"outputCount": 6,
			"outputBytes": 828
		}
	}],
	"remoteProcessGroupStatusList": null,
	"instanceStatus": null,
	"systemDiagnosticsStatus": null,
	"reportingTaskStatusList": null,
	"errorsGeneratingReport": []
}

I know minifi is still a small project i am so very grateful for the work the community is putting into it ! But is there a more detailed documentation around this ?

Thanks all

Don't have an account?
Coming from Hortonworks? Activate your account here