Support Questions

Find answers, ask questions, and share your expertise

Metron not indexing in elasticsearch

New Contributor

Hi all i am trying to acheive checkpoint logs into metron. I am using the following flow checkpoint --> nifi --> kafka -> logstash -> kafka (new topic) -> Logstash metron parser

Here are my files

Logstash conf

input {
 kafka{
   bootstrap_servers => "10.122.62.98:6667,10.122.62.99:6667"
   topics => "checkpoint"
   client_id => "checkpointwireless"
   tags => "checkpointfw"
}
}




filter {


if "checkpointfw" in [tags] {
syslog_pri { }


 if "checkpointfw" in [tags]  {
kv {
source => "message"
}


}
}
}




output {
if "checkpointfw" in [tags]   {
#stdout { codec => json_lines}
kafka {
 codec => rubydebug
 bootstrap_servers => "10.122.62.98:6667,10.122.62.99:6667"
 topic_id => "checkpoint_plain"
}
}
}


2 REPLIES 2

New Contributor

Here are my json files for indexing and parsing

{"parserClassName":"org.apache.metron.parsers.logstash.BasicLogstashParser","filterClassName":null,"sensorTopic":"checkpoint_wifi","writerClassName":null,"errorWriterClassName":null,"invalidWriterClassName":null,"readMetadata":false,"mergeMetadata":false,"numWorkers":null,"numAckers":null,"spoutParallelism":1,"spoutNumTasks":1,"parserParallelism":1,"parserNumTasks":1,"errorWriterParallelism":1,"errorWriterNumTasks":1,"spoutConfig":{},"securityProtocol":null,"stormConfig":{},"parserConfig":{},"fieldTransformations":[]}

New Contributor
{"hdfs":{"batchSize":1,"enabled":true,"index":"checkpoint"},"elasticsearch":{"batchSize":1,"enabled":true,"index":"checkpoint"},"solr":{"batchSize":1,"enabled":false,"index":"checkpoint_wifi"}}
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.