OK, havent read thre reference exactly enough. I should be able to use the existing date/time in the message field as an input for "convertTimestamp" to get a well formated RFC3339 timestamp. So, point 2 is partialy solve. How can i convert it to an unix teimstamp then?
... View more
Hi, I'm building the following setup for my central logging infrastructure: rSyslog Client ==> Flume Syslog Source ==> Memory Channel ==> Elastic Search Sink ==> ES Cluister <== Kibana 3 Web UI Unfortunately some vendors do not provide well formatted syslog messages. In my case the date/time is some kind of weird: 2013:09:17-09:03:03 ulogd: id="2001" severity="info" sys="SecureNet" foo="bar" ... I would like to use the Morphline interceptor to modify the date/time to a valid format and save it to the corresponding headers. So i use a simple "readLine" and "gork" to get out my fields (year, month, day, ...) as described in the manual/examples. Thats the easy part. But now I'm get stuck on how i can put the single fields together again: The single fields of the date/time should be converted to a timestamp and overwrite the existing header (@fields.timestamp) The timestamp should be converted to an RFC3339 format and overwrite the timestamp header of the flume syslog source (@timestamp) Don't know if this is possible. Maybe i should use NetCat Source instead and parse the whole message myself? The wrong date/time should be removed from the message because it's not needed there anymore Thank you very much for any help Urs
... View more