Created 07-20-2017 01:07 PM
Hello @H S,
Have you find a solution for Grok parsing Error :
java.lang.IllegalStateException: Grok parser Error: Grok statement produced a null message. Original message was: {"hostname":"my.machine","timestamp":"1490746284"} and the parsed message was: {}
I actually have the same error with a very simple Grok pattern :
PARSE_TXT %{DATA:email}\:%{DATA:name}
For the moment, i just try to parse the following text :
toto:toto tata:tata titi:titi
I have tested this on the grok constructor website: http://grokconstructor.appspot.com/ and it correctly match.
Anyone already had this error ?
My parser configuration :
{ "parserClassName": "org.apache.metron.parsers.GrokParser", "sensorTopic": "parse_txt", "parserConfig": { "grokPath": "/apps/metron/patterns/parse_txt", "patternLabel": "PARSE_TXT" } }
Thanks !
Created 07-20-2017 05:33 PM
No, I didn't find the root cause. But in my case, it was fairly easy to use the CSV parser instead of the Grok parser, with slight modifications to my telemetry source. You can find more details about the CSV parser here:
https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
Hope this helps.
Created 07-24-2017 09:08 AM
Hi @H S
Thanks for your answer, i have increased resources on my Full Dev VM and used Metron UI to create new sensor, currently it works.
But i have a new problem, Elasticsearch not index logs. I don't know if logs correctly reach Elasticsearch because in Storm UI, in parserBolt -> Output stats (for my topic), i don't have anythings. (Screen Below).
Has anyone ever had a problem with Kafka Stream ? Where logs correctly pass thought NiFi, correctly receive by Kafka and parse by Grok but are not redirect to Elasticsearch.
Created 04-06-2018 07:39 PM
I'm having a similar issue. In my case I'm feeding Yaf data to Kafka (it works correctly). Then it gives the parsing error:
java.lang.IllegalStateException: Grok parser Error: Grok statement produced a null message. Original message was:
I used the Metron's default YAF Grok pattern. I have checked the logs in http://grokconstructor.appspot.com and it is able to correctly match the pattern. Furthermore, I use the "Test" button on Metron's UI to check whether the patter is able to extract the fields from the log and it also works correctly.
Does anyone knows why this fails?
Example Log:
2018-04-04 10:05:32.533|2018-04-04 10:05:32.533| 0.000| 0.000| 6| 10.5.0.83| 2869| 10.5.0.97|52023| AR| 0| 0| 0|c019c454|00000000|000|000| 1| 40| 0| 0| 0|
Yaf Pattern
YAF_TIME_FORMAT %{YEAR:UNWANTED}-%{MONTHNUM:UNWANTED}-%{MONTHDAY:UNWANTED}[T ]%{HOUR:UNWANTED}:%{MINUTE:UNWANTED}:%{SECOND:UNWANTED}
YAF_DELIMITED %{YAF_TIME_FORMAT:start_time}\|%{YAF_TIME_FORMAT:end_time}\|%{SPACE:UNWANTED}%{BASE10NUM:duration}\|%{SPACE:UNWANTED}%{BASE10NUM:rtt}\|%{SPACE:UNWANTED}%{INT:protocol}\|%{SPACE:UNWANTED}%{IP:ip_src_addr}\|%{SPACE:UNWANTED}%{INT:ip_src_port}\|%{SPACE:UNWANTED}%{IP:ip_dst_addr}\|%{SPACE:UNWANTED}%{INT:ip_dst_port}\|%{SPACE:UNWANTED}%{DATA:iflags}\|%{SPACE:UNWANTED}%{DATA:uflags}\|%{SPACE:UNWANTED}%{DATA:riflags}\|%{SPACE:UNWANTED}%{DATA:ruflags}\|%{SPACE:UNWANTED}%{WORD:isn}\|%{SPACE:UNWANTED}%{DATA:risn}\|%{SPACE:UNWANTED}%{DATA:tag}\|%{GREEDYDATA:rtag}\|%{SPACE:UNWANTED}%{INT:pkt}\|%{SPACE:UNWANTED}%{INT:oct}\|%{SPACE:UNWANTED}%{INT:rpkt}\|%{SPACE:UNWANTED}%{INT:roct}\|%{SPACE:UNWANTED}%{INT:app}\|%{GREEDYDATA:end_reason}