Member since
03-01-2017
31
Posts
1
Kudos Received
0
Solutions
07-15-2017
02:26 AM
@Rahul P
You try to the guide bro plugin. But you change ["metadata.broker.list"] = "localhost:9092" to ["metadata.broker.list"] = "node1:6667"
... View more
07-14-2017
02:14 AM
Hi @Rahul P you can use bro-plugin-kafka for bro and flume for snort to forward events to metron.
... View more
06-29-2017
03:46 AM
Hi @asubramanian., If I start yaf by command: /etc/init.d/yaf start
And then I running command: nohub /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic yaf &
It's show log [2017-06-29 10:19:32,326] ERROR Error when sending message to topic yaf with key: null, value: 52 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Batch containing 190 record(s) expired due to timeout while requesting metadata from brokers for yaf-0
... View more
06-29-2017
02:46 AM
Hi @asubramanian,
When I run this command it have output. nohup /usr/local/bin/yaf --in eth0 --become-user yaf --live pcap | /usr/local/bin/yafscii --tabular
But when I run this command it don't have output nohup /usr/local/bin/yaf --in eth0 --become-user yaf --live pcap | /usr/local/bin/yafscii --tabular | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic yaf &
... View more
06-22-2017
08:02 AM
Hi @asubramanian, I try to follow your suggest but it's not ingest yaf.
... View more
06-10-2017
03:52 AM
1 Kudo
I configured Yaf on node instance. And I run command to ingest Yaf event to Kafka. When I check storm log on Metron node but I don't find yaf. nohup /usr/local/bin/yaf --in eth0 --become-user yaf --live pcap | /usr/local/bin/yafscii --tabular | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic yaf & Can you help me?
... View more
Labels:
- Labels:
-
Apache Metron
06-07-2017
10:06 AM
You check help me. please. 2017-06-07 17:09:32.589 o.a.m.p.s.BasicSnortParser [ERROR] Unable to parse message: [**] [1:10000001:1] ICMP test detected [**]
java.lang.IllegalArgumentException: Unexpected number of fields, expected: 27 in [**] [1:10000001:1] ICMP test detected [**]
at org.apache.metron.parsers.snort.BasicSnortParser.parse(BasicSnortParser.java:148) [stormjar.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:45) [stormjar.jar:?]
at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:123) [stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:466) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
2017-06-07 17:09:32.594 o.a.s.d.executor [ERROR]
java.lang.IllegalStateException: Unable to parse message: [**] [1:10000001:1] ICMP test detected [**]
at org.apache.metron.parsers.snort.BasicSnortParser.parse(BasicSnortParser.java:180) ~[stormjar.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:45) ~[stormjar.jar:?]
at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:123) [stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:466) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
Caused by: java.lang.IllegalArgumentException: Unexpected number of fields, expected: 27 in [**] [1:10000001:1] ICMP test detected [**]
at org.apache.metron.parsers.snort.BasicSnortParser.parse(BasicSnortParser.java:148) ~[stormjar.jar:?]
... 12 more
... View more
06-07-2017
09:47 AM
Hi @asubramanian I re-configured my snort system and It's show alert log. [**] [1:10000001:1] ICMP test detected [**]
[Classification: Generic ICMP event] [Priority: 3]
06/07/17-16:37:15.044404 172.16.1.10 -> 172.16.1.20
ICMP TTL:126 TOS:0x0 ID:14129 IpLen:20 DgmLen:60
Type:8 Code:0 ID:1 Seq:1523 ECHO
And I re-configured snort.json file {
"parserClassName":"org.apache.metron.parsers.snort.BasicSnortParser",
"sensorTopic":"snort",
"parserConfig": {
"dateFormat" : "MM/dd/yy-HH:mm:ss.SSSSSS",
"timeZone" : "America/New_York"
}
}
But it still fails.
... View more
06-07-2017
06:38 AM
I tested Snort alert and it's have log info following [**] [1:10000001:1] ICMP test detected [**]
[Classification: Generic ICMP event] [Priority: 3]
06/06-14:54:02.125421 172.16.1.10 -> 172.16.1.20
ICMP TTL:126 TOS:0x0 ID:15052 IpLen:20 DgmLen:60
Type:8 Code:0 ID:1 Seq:1473 ECHO
When I checked storm log and it's show 2017-06-07 09:39:41.083 o.a.s.d.executor [ERROR]
java.lang.IllegalStateException: Unable to parse message: 06/06-14:54:02.125421 172.16.1.10 -> 172.16.1.20
Can you help me?
... View more
- Tags:
- CyberSecurity
- Metron
Labels:
- Labels:
-
Apache Metron
06-05-2017
11:10 AM
I build YAF on node1 and Metron on node2. How to push YAF event to Metron? Can you suggest for me?
... View more
05-31-2017
04:30 AM
Hi @nallen Thank you. I install YAF, Snort on other nodes. How to push log data to Metron. Can you help me?
... View more
05-25-2017
08:14 AM
Hi @Jon Zeolla. Thank you for your answer. I set up separate servers Snort, YAF, Bro. How to I ingest log from that servers into metron?
... View more
05-24-2017
09:07 AM
Hi everyone. I configured Apache Metron full dev 0.3 but I don't push Snort, Bro, Yaf log to Metron Server. Can you help me?
... View more
Labels:
- Labels:
-
Apache Metron
04-18-2017
11:42 AM
When I check storm log, it's show. 2017-04-18 17:44:03.921 o.a.s.d.executor [ERROR]
java.lang.IllegalStateException: Unable to parse message: 04/18-17:44:00.620823 [**] [1:999158:0] Sample Metron Message from Snort [**] [Priority: 0]
... View more
04-18-2017
11:42 AM
Hi everyone., I install apache flume and Snort on Server. But Kibana don't index snort*. I configured flume.conf file. snort.channels=memory-channel
snort.channels.memory-channel.capacity=1000
snort.channels.memory-channel.transactionCapacity=100
snort.channels.memory-channel.type=memory
snort.sinks=kafka-sink logger-sink
snort.sinks.kafka-sink.brokerList=node1:6667
snort.sinks.kafka-sink.channel=memory-channel
snort.sinks.kafka-sink.topic=snort
snort.sinks.kafka-sink.type=org.apache.flume.sink.kafka.KafkaSink
snort.sinks.logger-sink.channel=memory-channel
snort.sinks.logger-sink.type=logger
snort.sources=exec-source
snort.sources.exec-source.channels=memory-channel
snort.sources.exec-source.command=tail -F /var/log/snort/alert
snort.sources.exec-source.logStdErr=true
snort.sources.exec-source.restart=true
snort.sources.exec-source.type=exec
... View more
04-11-2017
02:03 AM
Hi @asubramanian, Command "rwfilter --proto=0- --type=all --pass=stdout | rwcut | tail" showed results. But I can't push data from YAF Server to Metron.
... View more
04-08-2017
04:01 AM
Hi @asubramanian, Thank you very much. I'm have a problem. Wish you counseling help me. I configured YAF Server and I searched Google but I don't push YAF log to Metron.
... View more
04-07-2017
01:49 AM
Hi everyone. I deploy a Metron cluster using Ambari following article https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html. I pushed Bro logs to Kafka and It's show on Discover. But when I access http://metron:5000 then Kibana dashboard empty. Can you help me?
... View more
Labels:
- Labels:
-
Apache Metron
04-01-2017
07:25 AM
Hi @asubramanian I run command on YAF Server but Kibana not pattern index yaf_index*. Can you help me? nohup /usr/local/bin/yaf --silk --ipfix=tcp --live=pcap --out=node1 --ipfix-port=6667 --in=eth0 --applabel --max-payload=384 &
... View more
03-31-2017
02:48 AM
Hi @nallen. Thank you for your reply. I try to change "format": "strict_date_optional_time||epoch_millis" and delete index but Kibana don't display it properly.
... View more
03-31-2017
01:37 AM
Hi @asubramanian. Thank you for your reply. I done configured. Index pattern is bro_index_*
... View more
03-30-2017
10:10 AM
Hi everyone., I want Kibana to show datetime format standard. I push Bro log via Bro Kafka plugin but It show timestamp format. Can you help me?
... View more
Labels:
- Labels:
-
Apache Metron
03-30-2017
03:24 AM
I find on /usr/metron/0.3.0/patterns path but It's have asa, common, fireeye, sourcefire, squid, websphere, yaf files.
... View more
03-29-2017
01:43 AM
Hi @asubramanian, Thanks you. I done configured follow your guide, but I don't pattern Bro log on Kibana. Can you help me?
... View more
03-10-2017
08:08 AM
Hello @asubramanian I tried your guide. But I don't understand add services for master & slave nodes. Can you help me? Many thanks.
... View more
03-10-2017
07:47 AM
Hello @Jon Zeolla. I use CentOS 7. I will try your script on CentOS 6.8. How many node do build on your script?
... View more
03-01-2017
07:14 AM
Hi @asubramanian Thank you for answering my questions. I try configure follow your guide
... View more