Ingesting XML Telemetry in Metron

[#1]

Hi,

I was wondering about the best way to ingest logs in XML format into Metron. Parsing with Grok doesn't seem to be the way to go in this case. Remaining options seem to be to 1) either utilize NiFi to turn the XML into a format that Metron expects, 2) or develop a java parser for parsing XML, (that is Metron-288 issue which is not of high priority: https://issues.apache.org/jira/browse/METRON-288 ).

Are these two options the only possibilities?

[#2 · Accepted solution]

Metron supports 3 types of parsers: Grok, CSV and Java. For XML data Java is the best choice.

You can see example parsers in the Metron github:

https://github.com/apache/incubator-metron/tree/master/metron-platform/metron-parsers/src/main/java/...

You could also use Nifi to convert the XML to JSON and enqueue the events to the enrichment topic. Here are some articles about parsing XML logs with Nifi:

https://community.hortonworks.com/articles/25720/parsing-xml-logs-with-nifi-part-1-of-3.html