Member since
05-26-2020
20
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
245 | 06-05-2020 10:55 AM |
01-21-2021
09:17 AM
Hello, I have files that are in server A. I have the ListSFTP -> FetchSFTP processors setup in server B. My goal is to Fetch the files from server A, so that they are in the pipeline of server B, but after fetching the files, I want to move the files to another remote location, server C. Can anybody help me out with this? I feel like it's a simple solution that I'm missing..
... View more
Labels:
09-14-2020
10:40 AM
Hi Steven, I tried the method you linked but that didn't work. I also configured the SSLContextService with the keystore and truststore I got from Elasticsearch Cluster. But I get the following error now: Is there anyway to stop hostname verification? Or configure it so hostname verification works correctly? I have blocked out the full hostnames but I can assure you the hostname (atlas201.xx.xxx.xx) is the exact same as the specified CN (atlas.xx.xxx.xx)
... View more
09-11-2020
08:41 AM
Hello, I am trying to configure the StandardSSLContextService controller to work with either the PutElasticsearchHttpRecord or PutElasticsearchRecord processors. The Elasticsearch cluster has been secured by Opendistro for Elasticsearch I am able to do the following below successfully: openssl s_client -connect atlas201.cs:9200 -debug -state -cert esnode.pem -key esnode-key.pem -CAfile root-ca.pem But when I try those keys and certs in the StandardSSLContextService I am unable to get it to work. I am not too familiar with Opendistro as this was already a pre-configured cluster that I am working on. Could someone tell me what the following parameters need to be for me to successfully connect and ingest data into the ES cluster?
... View more
- Tags:
- ElasticSearch
Labels:
08-04-2020
07:35 AM
@mburgessany suggestions? at the moment I am using SplitRecord -> PutElasticsearchHttpRecord and splitting it into 100,000 but it's pretty slow in comparison
... View more
08-04-2020
07:32 AM
Did you ever find a workaround/solution for this?
... View more
07-28-2020
12:12 PM
I have a processor tree that I'm testing out with large json objects (1GB each). The tree is basically GetFile -> UpdateAttribute (add avro schema name) -> PutElasticsearchHttpRecord But PutElasticsearchHttpRecord gives me the following error (see screenshot) Can anybody tell me what's wrong? I have already assigned enough RAM in bootstrap.conf in NiFi: # JVM memory settings java.arg.2=-Xms16g java.arg.3=-Xmx32g
... View more
- Tags:
- ElasticSearch
Labels:
07-06-2020
10:15 AM
Is threat intelligence enrichment possible within NiFi itself? I already have GeoIPLookupRecord processor setup and I was wondering if there was a similar way to do any form of threat intelligence on the same data as well? I am trying not to use Metron since it has not worked for me at all. Any suggestions?
... View more
Labels:
07-05-2020
03:36 PM
I have NiFi set up to publish logs into Kafka topics. The publishing of the logs to Kafka topics is working fine, but when I create a sensor in Metron Management UI, it is not receiving any of the data that is being published into the kafka topics. I can see the logs as well when I use the kafka-console-consumer.sh script. When I create the metron sensor, it automatically creates a storm parser and I can see it in the Storm UI page. But I do not see any of the logs in Metron or Storm. Is this a kafkaSpout issue? Possible settings within Metron that need to be changed? What do I need to do here? (as you can see, there are 3 running sensors, but no throughput for the 3 kafka topics) Can somebody point me in the direction to a solution for this please?
... View more
07-02-2020
09:46 AM
I am sending relatively large files to GeoEnrichIPRecord processor that are in JSON format. So I have a JsonTreeReader -> AvroSchemaRegistry -> JsonRecordSetWriter A file that is about 5GB is taking around 5-10mins to process. Is there anything I can do to increase the processing time? I am trying to get 11TB of data processed, so I need the processing time down significantly. I have allocated 16GB to JVM memory in bootstrap.conf: # JVM memory settings java.arg.2=-Xms16g java.arg.3=-Xmx16g I have also changed the maximum timer driven thread count to: 128 (4x CPU cores) My server hardware specs are as follows: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz 32 Cores 192 GB RAM
... View more
Labels:
06-26-2020
08:22 AM
Were you able to get this resolved? I am having the same issue now
... View more
06-26-2020
08:10 AM
I have a TailFile -> PublishKafka processor in NiFi for squid logs. But when I push the logs to Kafka, they don't appear in Elasticsearch. They do show up in Kafka when I run the ./kafka-console-producer.sh script. Storm also sees the new topic I created in Kafka named "squid" But Elasticsearch/Kibana don't see any of the logs. Is Metron not receiving the logs from Kafka? When I check the sensor in Metron Management UI, it says the Kafka Topic is "emitting" so I assume that Metron does see the logs coming into Kafka? I am not sure why I don't see the logs in Kibana or when I curl -xget $ELASTICSEARCH:9200 My Kafka Broker, Elasticsearch, Kibana, Metron, Zookeeper are all on the same physical node. Can someone please assist with this?
... View more
06-22-2020
06:52 AM
Hello, I currently have files in paths that look like this: /feed/analytics/2020-06-04 /feed/analytics/2020-06-20 /feed/processing/2020-06-04 /feed/processing/2020-06-15 /feed/graphing/2020-06-04 And so on, and I am trying to use ListSFTP to list the files within these directories and FetchSFTP to fetch the files. I tried the following with ListSFTP but it failed: If someone could point me out to the right regex I would appreciate it. Thanks!
... View more
- Tags:
- NiFi
Labels:
06-15-2020
02:07 PM
Is it possible to geo enrich bro/zeek logs that are currently in this format? : timestamp | uid | orig_ip | orig_p | resp_ip | resp_p | protocol | service | duration If yes, then what is the easiest way to approach this? If not, am I supposed to convert each entry into a JSON object, and then use GeoEnrichIP? I was trying to do use the LookUpRecord processor as per the instructions here: https://community.cloudera.com/t5/Community-Articles/Geo-Enrich-NiFi-Provenance-Event-Data-using-LookupRecord/ta-p/247107 But I don't have a JSON object coming into it, so I don't know how to handle this. Could someone please help me out?
... View more
- Tags:
- enrichment
- NiFi
Labels:
06-08-2020
07:27 AM
Hello, I currently have about 15GB of data that is split into conn, weird, and notice log.gz files. I am trying to figure out how to stream this data into Metron. Currently I have a NiFi Processor setup using ListSFTP, FetchSFTP and then decompressing the files. I am not sure what to do next to get the data enriched through Metron. Am I supposed to create Kibana indexes for the logs? Please help me to proceed further with this. Thanks
... View more
06-05-2020
10:55 AM
1 Kudo
Figured it out using GetSFTP
... View more
06-05-2020
06:59 AM
I currently have NiFi setup and running on my master node. I have data coming in to my data node (node2). But I want to setup a NiFi processor to Get or Fetch the data from node2. Which processor do I use and how? Also I have data sorted in folders represented by date as name. So I have data within folders such as 2020-06-03 , 2020-06-04 Is there a way to process the data within these efficiently?
... View more
- Tags:
- NiFi
Labels:
06-01-2020
06:42 AM
1 Kudo
I would also like to add that for some reason when I did this with CentOS with GNOME desktop, this fails. But if I run a minimal install, this works. I'm not sure if this is because of some python libraries being downloaded for GNOME.. Thank you for getting me to the next steps!
... View more
05-27-2020
01:24 PM
..
[INFO] ------------------------------------------------------------------------
[INFO] Building metron-config 0.7.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ metron-config ---
[INFO] Deleting /root/metron/metron-interface/metron-config/node (includes = [], excludes = [])
[INFO] Deleting /root/metron/metron-interface/metron-config/node_modules (includes = [], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-config ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.3:prepare-agent (default) @ metron-config ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.3/org.jacoco.agent-0.8.3-runtime.jar=destfile=/root/metron/metron-interface/metron-config/target/jacoco.exec
[INFO]
[INFO] --- frontend-maven-plugin:1.3:install-node-and-npm (install node and npm) @ metron-config ---
[INFO] Installing node version v9.11.1
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/9.11.1/node-9.11.1-linux-x64.tar.gz into /root/metron/metron-interface/metron-config/node/tmp
[INFO] Copying node binary from /root/metron/metron-interface/metron-config/node/tmp/node-v9.11.1-linux-x64/bin/node to /root/metron/metron-interface/metron-config/node/node
[INFO] Installed node locally.
[INFO] Installing npm version 6.2.0
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/npm/6.2.0/npm-6.2.0.tar.gz into /root/metron/metron-interface/metron-config/node/node_modules
[INFO] Installed npm locally.
[INFO]
[INFO] --- frontend-maven-plugin:1.3:npm (npm ci) @ metron-config ---
[INFO] npm not inheriting proxy config from Maven
[INFO] Running 'npm ci' in /root/metron/metron-interface/metron-config
[INFO]
[INFO] > fsevents@1.2.4 install /root/metron/metron-interface/metron-config/node_modules/fsevents
[INFO] > node install
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/fsevents/install'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[INFO]
[INFO] > node-sass@4.9.3 install /root/metron/metron-interface/metron-config/node_modules/node-sass
[INFO] > node scripts/install.js
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/node-sass/scripts/install.js'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[INFO]
[INFO] > puppeteer@1.8.0 install /root/metron/metron-interface/metron-config/node_modules/puppeteer
[INFO] > node install.js
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/puppeteer/install.js'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 1
[ERROR] npm ERR! puppeteer@1.8.0 install: `node install.js`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the puppeteer@1.8.0 install script.
[ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
[ERROR]
[ERROR] npm ERR! A complete log of this run can be found in:
[ERROR] npm ERR! /root/.npm/_logs/2020-05-27T20_14_01_068Z-debug.log
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [ 0.867 s]
[INFO] metron-stellar ..................................... SUCCESS [ 0.057 s]
[INFO] stellar-common ..................................... SUCCESS [ 8.449 s]
[INFO] metron-analytics ................................... SUCCESS [ 0.016 s]
[INFO] metron-maas-common ................................. SUCCESS [ 0.461 s]
[INFO] metron-platform .................................... SUCCESS [ 0.015 s]
[INFO] metron-zookeeper ................................... SUCCESS [ 0.121 s]
[INFO] metron-test-utilities .............................. SUCCESS [ 0.590 s]
[INFO] metron-integration-test ............................ SUCCESS [ 6.782 s]
[INFO] metron-maas-service ................................ SUCCESS [ 0.687 s]
[INFO] metron-common ...................................... SUCCESS [ 2.304 s]
[INFO] metron-statistics .................................. SUCCESS [ 0.503 s]
[INFO] metron-common-streaming ............................ SUCCESS [ 0.006 s]
[INFO] metron-common-storm ................................ SUCCESS [ 0.292 s]
[INFO] metron-hbase ....................................... SUCCESS [ 0.475 s]
[INFO] metron-enrichment .................................. SUCCESS [ 0.055 s]
[INFO] metron-enrichment-common ........................... SUCCESS [ 1.591 s]
[INFO] metron-writer ...................................... SUCCESS [ 0.013 s]
[INFO] metron-writer-common ............................... SUCCESS [ 0.705 s]
[INFO] metron-writer-storm ................................ SUCCESS [ 0.668 s]
[INFO] metron-storm-kafka-override ........................ SUCCESS [ 0.150 s]
[INFO] metron-storm-kafka ................................. SUCCESS [ 0.302 s]
[INFO] metron-profiler-common ............................. SUCCESS [ 0.372 s]
[INFO] metron-profiler-client ............................. SUCCESS [ 0.891 s]
[INFO] metron-profiler-storm .............................. SUCCESS [ 0.368 s]
[INFO] metron-profiler-spark .............................. SUCCESS [ 1.248 s]
[INFO] metron-profiler-repl ............................... SUCCESS [ 0.205 s]
[INFO] metron-hbase-client ................................ SUCCESS [ 0.012 s]
[INFO] metron-enrichment-storm ............................ SUCCESS [ 0.465 s]
[INFO] metron-indexing .................................... SUCCESS [ 0.680 s]
[INFO] metron-solr ........................................ SUCCESS [ 1.021 s]
[INFO] metron-job ......................................... SUCCESS [ 0.218 s]
[INFO] metron-pcap ........................................ SUCCESS [ 1.105 s]
[INFO] metron-pcap-backend ................................ SUCCESS [ 0.544 s]
[INFO] metron-data-management ............................. SUCCESS [ 1.130 s]
[INFO] metron-parsing ..................................... SUCCESS [ 0.004 s]
[INFO] metron-parsers-common .............................. SUCCESS [ 0.531 s]
[INFO] metron-parsers ..................................... SUCCESS [ 0.693 s]
[INFO] metron-management .................................. SUCCESS [ 1.067 s]
[INFO] elasticsearch-shaded ............................... SUCCESS [ 0.265 s]
[INFO] metron-elasticsearch ............................... SUCCESS [ 0.796 s]
[INFO] metron-parsing-storm ............................... SUCCESS [ 0.634 s]
[INFO] metron-hbase-server ................................ SUCCESS [ 0.253 s]
[INFO] metron-deployment .................................. SUCCESS [ 0.003 s]
[INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [ 0.038 s]
[INFO] Metron Ambari Management Pack ...................... SUCCESS [ 0.131 s]
[INFO] metron-contrib ..................................... SUCCESS [ 0.005 s]
[INFO] metron-docker ...................................... SUCCESS [ 0.008 s]
[INFO] metron-performance ................................. SUCCESS [ 0.453 s]
[INFO] metron-interface ................................... SUCCESS [ 0.002 s]
[INFO] metron-config ...................................... FAILURE [ 12.594 s]
[INFO] metron-alerts ...................................... SKIPPED
[INFO] metron-rest-client ................................. SKIPPED
[INFO] metron-rest ........................................ SKIPPED
[INFO] site-book .......................................... SKIPPED
[INFO] 3rd party Functions (just for tests) ............... SKIPPED
[INFO] stellar-zeppelin ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51.478 s
[INFO] Finished at: 2020-05-27T16:14:01-04:00
[INFO] Final Memory: 156M/1300M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:npm (npm ci) on project metron-config: Failed to run task: 'npm ci' failed. (error code 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :metron-config Unable to get any of the single-node VMs to work. Updated npm, nodejs as well but still getting the same error..
... View more
Labels:
05-27-2020
12:53 PM
[INFO] Building metron-config 0.7.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ metron-config ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-config ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.3:prepare-agent (default) @ metron-config ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.3/org.jacoco.agent-0.8.3-runtime.jar=destfile=/root/metron/metron-interface/metron-config/target/jacoco.exec
[INFO]
[INFO] --- frontend-maven-plugin:1.3:install-node-and-npm (install node and npm) @ metron-config ---
[INFO] Installing node version v9.11.1
[INFO] Downloading https://nodejs.org/dist/v9.11.1/node-v9.11.1-linux-x64.tar.gz to /root/.m2/repository/com/github/eirslett/node/9.11.1/node-9.11.1-linux-x64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/9.11.1/node-9.11.1-linux-x64.tar.gz into /root/metron/metron-interface/metron-config/node/tmp
[INFO] Copying node binary from /root/metron/metron-interface/metron-config/node/tmp/node-v9.11.1-linux-x64/bin/node to /root/metron/metron-interface/metron-config/node/node
[INFO] Installed node locally.
[INFO] Installing npm version 6.2.0
[INFO] Downloading http://registry.npmjs.org/npm/-/npm-6.2.0.tgz to /root/.m2/repository/com/github/eirslett/npm/6.2.0/npm-6.2.0.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/npm/6.2.0/npm-6.2.0.tar.gz into /root/metron/metron-interface/metron-config/node/node_modules
[INFO] Installed npm locally.
[INFO]
[INFO] --- frontend-maven-plugin:1.3:npm (npm ci) @ metron-config ---
[INFO] npm not inheriting proxy config from Maven
[INFO] Running 'npm ci' in /root/metron/metron-interface/metron-config
[INFO]
[INFO] > fsevents@1.2.4 install /root/metron/metron-interface/metron-config/node_modules/fsevents
[INFO] > node install
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/fsevents/install'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[INFO]
[INFO] > node-sass@4.9.3 install /root/metron/metron-interface/metron-config/node_modules/node-sass
[INFO] > node scripts/install.js
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/node-sass/scripts/install.js'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[INFO]
[INFO] > puppeteer@1.8.0 install /root/metron/metron-interface/metron-config/node_modules/puppeteer
[INFO] > node install.js
[INFO]
[ERROR] internal/modules/cjs/loader.js:638
[ERROR] throw err;
[ERROR] ^
[ERROR]
[ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/puppeteer/install.js'
[ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
[ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25)
[ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[ERROR] at startup (internal/bootstrap/node.js:283:19)
[ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 1
[ERROR] npm ERR! puppeteer@1.8.0 install: `node install.js`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the puppeteer@1.8.0 install script.
[ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
[ERROR]
[ERROR] npm ERR! A complete log of this run can be found in:
[ERROR] npm ERR! /root/.npm/_logs/2020-05-27T19_33_40_941Z-debug.log
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [ 8.174 s]
[INFO] metron-stellar ..................................... SUCCESS [ 0.561 s]
[INFO] stellar-common ..................................... SUCCESS [ 43.744 s]
[INFO] metron-analytics ................................... SUCCESS [ 0.140 s]
[INFO] metron-maas-common ................................. SUCCESS [ 0.707 s]
[INFO] metron-platform .................................... SUCCESS [ 3.832 s]
[INFO] metron-zookeeper ................................... SUCCESS [ 0.298 s]
[INFO] metron-test-utilities .............................. SUCCESS [ 0.489 s]
[INFO] metron-integration-test ............................ SUCCESS [ 6.854 s]
[INFO] metron-maas-service ................................ SUCCESS [ 4.830 s]
[INFO] metron-common ...................................... SUCCESS [ 18.655 s]
[INFO] metron-statistics .................................. SUCCESS [ 15.485 s]
[INFO] metron-common-streaming ............................ SUCCESS [ 0.095 s]
[INFO] metron-common-storm ................................ SUCCESS [ 0.896 s]
[INFO] metron-hbase ....................................... SUCCESS [ 5.999 s]
[INFO] metron-enrichment .................................. SUCCESS [ 0.178 s]
[INFO] metron-enrichment-common ........................... SUCCESS [ 28.021 s]
[INFO] metron-writer ...................................... SUCCESS [ 0.102 s]
[INFO] metron-writer-common ............................... SUCCESS [ 14.084 s]
[INFO] metron-writer-storm ................................ SUCCESS [ 0.922 s]
[INFO] metron-storm-kafka-override ........................ SUCCESS [ 6.694 s]
[INFO] metron-storm-kafka ................................. SUCCESS [ 0.457 s]
[INFO] metron-profiler-common ............................. SUCCESS [ 0.913 s]
[INFO] metron-profiler-client ............................. SUCCESS [ 19.419 s]
[INFO] metron-profiler-storm .............................. SUCCESS [ 44.677 s]
[INFO] metron-profiler-spark .............................. SUCCESS [02:20 min]
[INFO] metron-profiler-repl ............................... SUCCESS [ 13.246 s]
[INFO] metron-hbase-client ................................ SUCCESS [ 6.349 s]
[INFO] metron-enrichment-storm ............................ SUCCESS [ 49.006 s]
[INFO] metron-indexing .................................... SUCCESS [ 21.283 s]
[INFO] metron-solr ........................................ SUCCESS [01:27 min]
[INFO] metron-job ......................................... SUCCESS [ 0.424 s]
[INFO] metron-pcap ........................................ SUCCESS [ 1.544 s]
[INFO] metron-pcap-backend ................................ SUCCESS [ 25.809 s]
[INFO] metron-data-management ............................. SUCCESS [ 26.077 s]
[INFO] metron-parsing ..................................... SUCCESS [ 0.077 s]
[INFO] metron-parsers-common .............................. SUCCESS [ 49.663 s]
[INFO] metron-parsers ..................................... SUCCESS [ 44.046 s]
[INFO] metron-management .................................. SUCCESS [ 7.935 s]
[INFO] elasticsearch-shaded ............................... SUCCESS [ 36.180 s]
[INFO] metron-elasticsearch ............................... SUCCESS [ 46.452 s]
[INFO] metron-parsing-storm ............................... SUCCESS [ 46.663 s]
[INFO] metron-hbase-server ................................ SUCCESS [ 10.589 s]
[INFO] metron-deployment .................................. SUCCESS [ 0.087 s]
[INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [ 0.107 s]
[INFO] Metron Ambari Management Pack ...................... SUCCESS [ 0.373 s]
[INFO] metron-contrib ..................................... SUCCESS [ 0.065 s]
[INFO] metron-docker ...................................... SUCCESS [ 0.486 s]
[INFO] metron-performance ................................. SUCCESS [ 2.768 s]
[INFO] metron-interface ................................... SUCCESS [ 0.085 s]
[INFO] metron-config ...................................... FAILURE [ 40.080 s]
[INFO] metron-alerts ...................................... SKIPPED
[INFO] metron-rest-client ................................. SKIPPED
[INFO] metron-rest ........................................ SKIPPED
[INFO] site-book .......................................... SKIPPED
[INFO] 3rd party Functions (just for tests) ............... SKIPPED
[INFO] stellar-zeppelin ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14:50 min
[INFO] Finished at: 2020-05-27T15:33:40-04:00
[INFO] Final Memory: 218M/2196M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:npm (npm ci) on project metron-config: Failed to run task: 'npm ci' failed. (error code 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :metron-config Here is the output of platform-info.sh script Metron 0.7.1
fatal: Not a git repository (or any of the parent directories): .git
--
ansible 2.9.9
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /bin/ansible
python version = 2.7.5 (default, Apr 2 2020, 13:16:51) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
--
Vagrant 2.2.9
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.5
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00)
Maven home: /opt/apache-maven-3.3.9
Java version: 1.8.0_252, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1127.8.2.el7.x86_64", arch: "amd64", family: "unix"
--
Docker version 1.13.1, build 64e9980/1.13.1
--
node
v10.20.1
--
npm
6.14.4
--
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--
Compiler is C++11 compliant
--
Linux scslt227 3.10.0-1127.8.2.el7.x86_64 #1 SMP Tue May 12 16:57:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 19681.2 MB
Processor Model: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
Processor Speed: 3099.957 MHz
Total Physical Processors: 4
Total cores: 8
Disk information:
/dev/mapper/centos-root 66G 15G 52G 22% /
/dev/nvme0n1p5 1014M 227M 788M 23% /boot
/dev/nvme0n1p1 200M 12M 189M 6% /boot/efi
/dev/mapper/centos-home 21G 360M 21G 2% /home
This CPU appears to support virtualization I have also tried the single-node VM using vagrant and it also stops at metron-config. Could anyone help with this? I've been trying for quite a few days now scouring through the forums..
... View more
Labels: