Created on 12-17-2020 11:42 AM - last edited on 12-17-2020 10:16 PM by VidyaSargur
I want to parse application log data having format: <date> key1="value1" key2="value2" .... and generate output as json.
And this log data does not follow any pattern, and these key value pairs are dynamic. So I am looking for something similar to KV filter in logstash, where I can get all the key values from log.
Not sure how to proceed with this. Appreciate a quick help here.
Created 12-17-2020 11:06 PM
hello @spa
I've been looking for this also but it doesn't exist. Then, you can use a script (python, groovy,...) In case you have performance issue with scriptprocessor, you can improve the situation using the trick here:
Created 02-04-2021 12:19 PM
Hello @mburgess
Can you please answer the above question from VidyaSargur?
Created 02-09-2021 02:12 PM
If you use GrokReader you can use the same kv filter from logstash: https://community.cloudera.com/t5/Support-Questions/Grok-Patterns-Expressions-for-capturing-comma-se...
Created 02-14-2021 12:23 AM
Hi @mburgess
Can you please elaborate on which property needs to be configured in the GrokReader controller service for using the kv filter?
I'm trying to parse the incoming key=value pair.
Input: key1=value1,key2=value2,key3=value3,key4=value4
output: I need key1, key2, key3, key4 as attributes and their respective values as attribute values
I can use %{GREEDYDATA:msgbody} in the GrokExpression property but I do not know where to provide
kv {
source = "msgbody"
}
Your help is appreciated