SysDig
SysDig (Github) is an open source tool that allows for the exploration, analysis and trouble shooting of Linux systems and containers. It is well documented and very easy to install and use. It can be used for container and Linux system diagnostics, security analysis, monitoring and basic system information capture. Remember that sysdig can produces thousands of lines of messages and can continue doing so forever depending on the options selected. Check out the examples and read through all the options, you can monitor a ton of data really fast and also check for security anomalies.
NiFi Ingesting SysDig
Sysdig can produce amazing amounts of logs. I chose to ingest 1 second chunks as ASCII JSON. I selected those options and listed them below. The results are arrays of JSON. I decided it's best to save them as a large JSON files for now and convert them to ORC later for Hive analysis in Zeppelin. You could also split them into individual JSON rows and process those. I also save them to Apache Phoenix for fast queries.
ExecuteProcess Command
sysdig -A -j -M 1 --unbuffered
I just wrap that in a shell script for neatness.
HDF 2.0 / NIFI 1.0.0 Flow
Event JSON from SysDig
{"evt.cpu":6,"evt.dir":">","evt.info":"fd=7(<f>/usr/lib64/python2.7/lib-dynload/_elementtree.so) ","evt.num":111138,"evt.outputtime":1477313882635597873,"evt.type":"fstat",
"proc.name":"python","thread.tid":14602}
Apache Phoenix Table
CREATE TABLE sysdigevents ( evtcpu varchar, evtdir varchar, evtinfo varchar, evtoutputtime varchar, evttype varchar, procname varchar, threadtid varchar, evtnum varchar not null primary key );
Links
NiFi Flow
