Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Apache minifi Monitoring

Highlighted

Apache minifi Monitoring

Contributor

Hi all,

I am trying to understand the output of the minifi.sh flowfile <options> commands.

I wanna be able to monitor protectively all the "edge points" created by minifi.

Example:

The is the output of this cmd

/minifi.sh  flowStatus systemdiagnostics:heap,processorstats,contentrepositoryusage,flowfilerepositoryusage,garbagecollection

Output

{
	"controllerServiceStatusList": null,
	"processorStatusList": null,
	"connectionStatusList": null,
	"remoteProcessGroupStatusList": null,
	"instanceStatus": null,
	"systemDiagnosticsStatus": {
		"garbageCollectionStatusList": [{
			"name": null,
			"collectionCount": 3067,
			"collectionTime": 26764
		},
		{
			"name": null,
			"collectionCount": 6,
			"collectionTime": 861
		}],
		"heapStatus": {
			"totalHeap": 260571136,
			"maxHeap": 260571136,
			"freeHeap": 100187704,
			"usedHeap": 160383432,
			"heapUtilization": 62,
			"totalNonHeap": 76414976,
			"maxNonHeap": -1,
			"freeNonHeap": 2070192,
			"usedNonHeap": 74344784,
			"nonHeapUtilization": -1
		},
		"contentRepositoryUsageList": [{
			"name": "default",
			"freeSpace": 11783532544,
			"totalSpace": 29994987520,
			"usedSpace": 18211454976,
			"diskUtilization": 61
		}],
		"flowfileRepositoryUsage": {
			"freeSpace": 11783532544,
			"totalSpace": 29994987520,
			"usedSpace": 18211454976,
			"diskUtilization": 61
		},
		"processorStatus": {
			"loadAverage": 10.52,
			"availableProcessors": 14
		}
	},
	"reportingTaskStatusList": null,
	"errorsGeneratingReport": []
}

Some of the metrics listed are self explanatory but not in all cases and i would hate to make wrong assumptions and build a process around wrong interpreted metrics.

Where do i get stuck ?!

- trying to capture connection metrics/values

/minifi.sh flowStatus connection:all:health,stats

And i have Questions like :

connextionstatus.inputCount: 25 <--- this value is this since begging ? is this based on 5 min interval ? How to i interpret this ?

{
	"controllerServiceStatusList": null,
	"processorStatusList": null,
	"connectionStatusList": [{
		"name": "OUT Monitor - Clean and Extract//015f11b2-ba5c-14f0-d0dd-f17188d247d8",
		"connectionHealth": {
			"queuedCount": 0,
			"queuedBytes": 0
		},
		"connectionStats": {
			"inputCount": 25,
			"inputBytes": 3584,
			"outputCount": 25,
			"outputBytes": 3584
		}
	},
	{
		"name": "OUT Monitor - Initiate //IN Monitor - Clean and Extract",
		"connectionHealth": {
			"queuedCount": 0,
			"queuedBytes": 0
		},
		"connectionStats": {
			"inputCount": 6,
			"inputBytes": 828,
			"outputCount": 6,
			"outputBytes": 828
		}
	}],
	"remoteProcessGroupStatusList": null,
	"instanceStatus": null,
	"systemDiagnosticsStatus": null,
	"reportingTaskStatusList": null,
	"errorsGeneratingReport": []
}

I know minifi is still a small project i am so very grateful for the work the community is putting into it ! But is there a more detailed documentation around this ?

Thanks all

Don't have an account?
Coming from Hortonworks? Activate your account here