Member since
06-02-2017
30
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1990 | 12-17-2018 07:35 PM |
08-13-2019
02:27 PM
Hi @Michael Hidalgo , actually, I pushed the last commit to that file ( cleaning up errors in builds ). Casey Stella is actually the person who is the SME with that code, he wrote the doc. I would suggest sending an email to the dev@metron.apache.org list. You can subscribe by sending a mail to dev-subscribe@metron.apache.org. More people will see the question there. I assume you are running the stellar shell on a cluster machine, and logged in as an account that has rights to that HDFS location right? check that, but let's move that to the mailing list.
... View more
12-17-2018
07:35 PM
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mongodb-client-service-api</artifactId>
<scope>provided</scope>
</dependency> Your dependency should be that. You also need to have a NAR dependency <packaging>nar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mongodb-client-service-api-nar</artifactId>
<version>1.8.0</version>
<type>nar</type>
</dependency>
... View more
11-05-2018
06:21 PM
You changed the what to centos 7? The image that metron builds into? That isn't going to work. Apache Metron only supports centos6 for the vagrant - ansible build.
... View more
09-28-2018
06:32 PM
I don't know how you are doing docker, or if you are using HDF or just apache nifi, but Apache nifi is put in /opt/nifi https://github.com/apache/nifi/blob/master/nifi-docker/dockermaven/Dockerfile
... View more
09-18-2018
11:43 AM
1 Kudo
Can you build manually? Check the readme in the rpm project and try to build it, you will get the std out of the rpm build and there will be more info
... View more
09-07-2018
11:10 AM
Failing in metron-rpm usually means that docker is not running
... View more
08-27-2018
06:34 PM
https://github.com/apache/metron/pull/1175 Supports RFC 5424 messages only
... View more
07-15-2018
03:11 PM
https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html
... View more
04-17-2018
12:03 AM
InvokeHttp should be preferred @Damon Jones
... View more
04-12-2018
06:48 PM
The bug you see in your log was just fixed in master: https://github.com/apache/metron/pull/991
... View more
04-12-2018
06:47 PM
By default the vagrant vm doesn't have enough storm slots open to add new parsers to it. You are going to want to verify that, and either stop other parsers or add slots in the storm configuration in ambari.
... View more
04-09-2018
01:40 PM
1 Kudo
Also, can you create and build one from the command line without eclipse?
... View more
04-06-2018
07:20 PM
try $.[*] or $[*] http://jsonpath.herokuapp.com/ is a good place to practice https://github.com/json-path/JsonPath for documentation
... View more
02-06-2018
07:43 PM
I ran your message through the integration test code, and it worked. After changing the grok, you have to restart the parser topology in storm.
... View more
02-06-2018
01:08 PM
In the Apache Metron codebase, the squid grok rules are as such, and produce a timestamp without the .NN. SQUID_DELIMITED %{NUMBER:timestamp}[^0-9]*%{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url}[^0-9]*(%{IP:ip_dst_addr})?
Can you try that?
... View more
02-06-2018
01:04 PM
I don't understand what you mean by "add storm config directly". Can you put the steps you take? Are you saying if you take those steps everything works correctly?
... View more
02-05-2018
08:30 PM
Yes, not validating will fail the message. The Grok Parser will not validate if it doesn't find a "timestamp" field that has a > 0 value. @Override
public boolean validate(JSONObject message) {
LOG.debug("Grok parser validating message: {}", message);
Object timestampObject = message.get(Constants.Fields.TIMESTAMP.getName());
if (timestampObject instanceof Long) {
Long timestamp = (Long) timestampObject;
if (timestamp > 0) {
LOG.debug("Grok parser validated message: {}", message);
return true;
}
}
LOG.debug("Grok parser did not validate message: {}", message);
return false;
}
... View more
12-04-2017
03:11 PM
Could you post a screenshot of just the ambari screen?
... View more
12-04-2017
12:00 PM
From the ambari main screen, when you click on the 'red' metron service you should go to a page that lists the service components, what do you see there? Also in the top left of the page is the alerts area, when you click on that you should be able to get some detail about what is happening.
... View more
11-27-2017
06:14 PM
1 Kudo
We are seeing issues when gcc-c++ doesn't support cpp 11, ie gcc-c++ < 4.8
... View more
09-22-2017
10:42 AM
As far as I can tell, this is the only documentation. There is no per adapter document.
... View more
09-21-2017
06:44 PM
https://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html
... View more
08-09-2017
08:17 PM
Also, to follow on to Matt's answer: If you add your parser this way, then you will have to maintain your own fork of metron, and manage syncing it with apache/metron/master or release. Also, I think it is only implied by Matt, so I will explicitly point out that you will need to build the metron product yourself, and then do installs with YOUR built version of metron to get your parser installed. If not, you will have to do manual steps, one of which will include 'replace the metron-parsers-XXXX.jar on ALL the metron cluster nodes with yours'. METRON-777 provides the foundation for the extension system, and moves the existing parsers over to it. It also introduces the maven archetype. The follow on METRON-942 however provides the REST api support for installing and uninstalling parsers that you create. After 942 lands, your use case will be: * install the mvn archetype * create a directory * run the archetype * re-write the sample parser or replace it ( and the provided unit and integration test/configuration ) * build it * go to the rest page and install it * create a new kafka topic ( rest again ) * create a new storm topology for it ?? ?? Profit.
... View more
08-09-2017
08:08 PM
The CONFIG_GET function is part of the metron-management package, which up until a certain commit was NOT being installed when we installed metron. If you check the README.md under /metron-system/metron-management for the tag you are running you will see the instructions.
... View more
07-27-2017
11:11 PM
@ppp rrr -> you need to RUN the platform-info.sh. We have our own copy 😉 Also -> In ambari the Alerts at the top leftish will have the actual errors from the start operation.
... View more
07-24-2017
08:06 PM
I would suggest: Go into /etc/hosts and remove any node1 entry you see go into the quickdev dir and vagrant destroy go into the full dev dir and vagrant destroy then in full dev vagrant up
... View more
07-24-2017
07:29 PM
It is because you are doing sudo there. you do not have to sudo to run vagrant up.
... View more
06-24-2017
02:52 PM
I believe the python version should be 2.7.11 +
... View more
06-11-2017
10:49 AM
Metron Releases Github is not the release venue for Apache projects. Metron 0.4.0 has NOT been release yet. The existence of a release branch does not designate that an official release, although it is made in preparation for such a release. Once the release branch has been created, there may still be changes as the release goes through further testing, or as issues are found that need to be addressed. Creating the release branch allows the project to continue progressing and accepting pull requests on master, while isolating the release, and accepting only critical, release blocking changes. You are welcome to join the Metron Community and track the release, as all Metron business happens openly on the list ( such as voting for releases ) as is the Apache way.
... View more
06-03-2017
03:41 AM
When you say metron stops.... what do you mean specifically? If you mean the metron service in ambari, when you click on the service do you see which components have failed? Do you have alerts in ambari?
... View more