Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Event Correlation and Anomaly Detection in metron

avatar
Rising Star

hello to all,

I have reviewed metron docs and it's been indicated (for many times) that telemetry correlation and anomaly detection are two of metron main tasks.

Now i need to know which components do these tasks. I'm interested to see the source code doing correlation & anomaly detection.

Has anyone any idea?does anybody know where can I find them?

Thanks in advance.

1 ACCEPTED SOLUTION

avatar
Guru

There are a variety of meanings of correlation in Metron.

One means is adding correlation keys in the enrichment process, which then allow you to view events together once they are indexed. So you can correlate events by adding a common search key you can then pivot on in Kibana. This is a great means of investigating correlations between alerts and events.

For a more statistical approach to correlation, you will want to look into the profiler which maintains windows of data, which can then be used to correlate time series data using, for example, arima in a model managed by the model as a service infrastructure. This area of Metron is growing quite fast at the moment. I would suggest also looking at the Stellar statistics functions which can be used to build simple anomaly based models as well. It's also easy enough to add functions to Stellar if you want to extend the functionality.

View solution in original post

4 REPLIES 4

avatar
Guru

There are a variety of meanings of correlation in Metron.

One means is adding correlation keys in the enrichment process, which then allow you to view events together once they are indexed. So you can correlate events by adding a common search key you can then pivot on in Kibana. This is a great means of investigating correlations between alerts and events.

For a more statistical approach to correlation, you will want to look into the profiler which maintains windows of data, which can then be used to correlate time series data using, for example, arima in a model managed by the model as a service infrastructure. This area of Metron is growing quite fast at the moment. I would suggest also looking at the Stellar statistics functions which can be used to build simple anomaly based models as well. It's also easy enough to add functions to Stellar if you want to extend the functionality.

avatar
Rising Star

@Simon Elliston Ball @Timothy Spann

thank you both for your helpful answers, actually it took me a while to go through your links, but now I know what I needed.

avatar
Explorer

Could you elaborate on your findings please ? How can I trigger a complex alarm involving simple alarms from different logs ? Is Stellar of any help for it ?