question Re: Metron Build fails on metron config - single node vagrant VM in Support Questions

Metron Build fails on metron config - single node vagrant VM

HarshR — Wed, 27 May 2020 20:24:27 GMT

.. [INFO] ------------------------------------------------------------------------ [INFO] Building metron-config 0.7.1 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ metron-config --- [INFO] Deleting /root/metron/metron-interface/metron-config/node (includes = [], excludes = []) [INFO] Deleting /root/metron/metron-interface/metron-config/node_modules (includes = [], excludes = []) [INFO] [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-config --- [INFO] [INFO] --- jacoco-maven-plugin:0.8.3:prepare-agent (default) @ metron-config --- [INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.3/org.jacoco.agent-0.8.3-runtime.jar=destfile=/root/metron/metron-interface/metron-config/target/jacoco.exec [INFO] [INFO] --- frontend-maven-plugin:1.3:install-node-and-npm (install node and npm) @ metron-config --- [INFO] Installing node version v9.11.1 [INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/9.11.1/node-9.11.1-linux-x64.tar.gz into /root/metron/metron-interface/metron-config/node/tmp [INFO] Copying node binary from /root/metron/metron-interface/metron-config/node/tmp/node-v9.11.1-linux-x64/bin/node to /root/metron/metron-interface/metron-config/node/node [INFO] Installed node locally. [INFO] Installing npm version 6.2.0 [INFO] Unpacking /root/.m2/repository/com/github/eirslett/npm/6.2.0/npm-6.2.0.tar.gz into /root/metron/metron-interface/metron-config/node/node_modules [INFO] Installed npm locally. [INFO] [INFO] --- frontend-maven-plugin:1.3:npm (npm ci) @ metron-config --- [INFO] npm not inheriting proxy config from Maven [INFO] Running 'npm ci' in /root/metron/metron-interface/metron-config [INFO] [INFO] > fsevents@1.2.4 install /root/metron/metron-interface/metron-config/node_modules/fsevents [INFO] > node install [INFO] [ERROR] internal/modules/cjs/loader.js:638 [ERROR] throw err; [ERROR] ^ [ERROR] [ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/fsevents/install' [ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15) [ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25) [ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12) [ERROR] at startup (internal/bootstrap/node.js:283:19) [ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3) [INFO] [INFO] > node-sass@4.9.3 install /root/metron/metron-interface/metron-config/node_modules/node-sass [INFO] > node scripts/install.js [INFO] [ERROR] internal/modules/cjs/loader.js:638 [ERROR] throw err; [ERROR] ^ [ERROR] [ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/node-sass/scripts/install.js' [ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15) [ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25) [ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12) [ERROR] at startup (internal/bootstrap/node.js:283:19) [ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3) [INFO] [INFO] > puppeteer@1.8.0 install /root/metron/metron-interface/metron-config/node_modules/puppeteer [INFO] > node install.js [INFO] [ERROR] internal/modules/cjs/loader.js:638 [ERROR] throw err; [ERROR] ^ [ERROR] [ERROR] Error: Cannot find module '/root/metron/metron-interface/metron-config/node_modules/puppeteer/install.js' [ERROR] at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15) [ERROR] at Function.Module._load (internal/modules/cjs/loader.js:562:25) [ERROR] at Function.Module.runMain (internal/modules/cjs/loader.js:831:12) [ERROR] at startup (internal/bootstrap/node.js:283:19) [ERROR] at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3) [ERROR] npm ERR! code ELIFECYCLE [ERROR] npm ERR! errno 1 [ERROR] npm ERR! puppeteer@1.8.0 install: `node install.js` [ERROR] npm ERR! Exit status 1 [ERROR] npm ERR! [ERROR] npm ERR! Failed at the puppeteer@1.8.0 install script. [ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above. [ERROR] [ERROR] npm ERR! A complete log of this run can be found in: [ERROR] npm ERR! /root/.npm/_logs/2020-05-27T20_14_01_068Z-debug.log [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Metron ............................................. SUCCESS [ 0.867 s] [INFO] metron-stellar ..................................... SUCCESS [ 0.057 s] [INFO] stellar-common ..................................... SUCCESS [ 8.449 s] [INFO] metron-analytics ................................... SUCCESS [ 0.016 s] [INFO] metron-maas-common ................................. SUCCESS [ 0.461 s] [INFO] metron-platform .................................... SUCCESS [ 0.015 s] [INFO] metron-zookeeper ................................... SUCCESS [ 0.121 s] [INFO] metron-test-utilities .............................. SUCCESS [ 0.590 s] [INFO] metron-integration-test ............................ SUCCESS [ 6.782 s] [INFO] metron-maas-service ................................ SUCCESS [ 0.687 s] [INFO] metron-common ...................................... SUCCESS [ 2.304 s] [INFO] metron-statistics .................................. SUCCESS [ 0.503 s] [INFO] metron-common-streaming ............................ SUCCESS [ 0.006 s] [INFO] metron-common-storm ................................ SUCCESS [ 0.292 s] [INFO] metron-hbase ....................................... SUCCESS [ 0.475 s] [INFO] metron-enrichment .................................. SUCCESS [ 0.055 s] [INFO] metron-enrichment-common ........................... SUCCESS [ 1.591 s] [INFO] metron-writer ...................................... SUCCESS [ 0.013 s] [INFO] metron-writer-common ............................... SUCCESS [ 0.705 s] [INFO] metron-writer-storm ................................ SUCCESS [ 0.668 s] [INFO] metron-storm-kafka-override ........................ SUCCESS [ 0.150 s] [INFO] metron-storm-kafka ................................. SUCCESS [ 0.302 s] [INFO] metron-profiler-common ............................. SUCCESS [ 0.372 s] [INFO] metron-profiler-client ............................. SUCCESS [ 0.891 s] [INFO] metron-profiler-storm .............................. SUCCESS [ 0.368 s] [INFO] metron-profiler-spark .............................. SUCCESS [ 1.248 s] [INFO] metron-profiler-repl ............................... SUCCESS [ 0.205 s] [INFO] metron-hbase-client ................................ SUCCESS [ 0.012 s] [INFO] metron-enrichment-storm ............................ SUCCESS [ 0.465 s] [INFO] metron-indexing .................................... SUCCESS [ 0.680 s] [INFO] metron-solr ........................................ SUCCESS [ 1.021 s] [INFO] metron-job ......................................... SUCCESS [ 0.218 s] [INFO] metron-pcap ........................................ SUCCESS [ 1.105 s] [INFO] metron-pcap-backend ................................ SUCCESS [ 0.544 s] [INFO] metron-data-management ............................. SUCCESS [ 1.130 s] [INFO] metron-parsing ..................................... SUCCESS [ 0.004 s] [INFO] metron-parsers-common .............................. SUCCESS [ 0.531 s] [INFO] metron-parsers ..................................... SUCCESS [ 0.693 s] [INFO] metron-management .................................. SUCCESS [ 1.067 s] [INFO] elasticsearch-shaded ............................... SUCCESS [ 0.265 s] [INFO] metron-elasticsearch ............................... SUCCESS [ 0.796 s] [INFO] metron-parsing-storm ............................... SUCCESS [ 0.634 s] [INFO] metron-hbase-server ................................ SUCCESS [ 0.253 s] [INFO] metron-deployment .................................. SUCCESS [ 0.003 s] [INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [ 0.038 s] [INFO] Metron Ambari Management Pack ...................... SUCCESS [ 0.131 s] [INFO] metron-contrib ..................................... SUCCESS [ 0.005 s] [INFO] metron-docker ...................................... SUCCESS [ 0.008 s] [INFO] metron-performance ................................. SUCCESS [ 0.453 s] [INFO] metron-interface ................................... SUCCESS [ 0.002 s] [INFO] metron-config ...................................... FAILURE [ 12.594 s] [INFO] metron-alerts ...................................... SKIPPED [INFO] metron-rest-client ................................. SKIPPED [INFO] metron-rest ........................................ SKIPPED [INFO] site-book .......................................... SKIPPED [INFO] 3rd party Functions (just for tests) ............... SKIPPED [INFO] stellar-zeppelin ................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 51.478 s [INFO] Finished at: 2020-05-27T16:14:01-04:00 [INFO] Final Memory: 156M/1300M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:npm (npm ci) on project metron-config: Failed to run task: 'npm ci' failed. (error code 1) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :metron-config

Unable to get any of the single-node VMs to work. Updated npm, nodejs as well but still getting the same error..

Re: Metron Build fails on metron config - single node vagrant VM

stevenmatison — Thu, 28 May 2020 13:10:06 GMT

@HarshR I was able to build without issues in a single vagrant VM which I use for build from source projects. I suspect you need a slight modification to your build environment. I have shared all the details of mine below.

Here are the commands I used:

  1  git clone --recursive https://github.com/apache/metron
  2  cd metron/
  3  mvn clean install -DskipTests

Here are the changes in the Operating System:

    1  yum install wget rsync nano maven java java-devel rpm-build gcc -y
    2  yum install ntp python python-devel rpm-build gcc-c++ java-1.7.0-openjdk java-1.7.0-openjdk-devel git -y
    3  wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea
    4  sh setuptools-0.6c11-py2.7.egg
    5  wget https://downloads.apache.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz -P /tmp
    6  tar xf /tmp/apache-maven-3.6.3-bin.tar.gz -C /opt
    7  ln -s /opt/apache-maven-3.6.3 /opt/maven
    8  nano /etc/profile.d/maven.sh
    9  chmod +x /etc/profile.d/maven.sh
   10  source /etc/profile.d/maven.sh
   28  yum --enablerepo=extras install epel-release
   29  yum install npm -y

The setuptools and custom maven version work were to satisfy building Ambari From Source. The contents of the maven file above:

export JAVA_HOME=/usr/lib/jvm/jre-openjdk
export M2_HOME=/opt/maven
export MAVEN_HOME=/opt/maven
export PATH=${M2_HOME}/bin:${PATH}

Versions:

    npm: 3.10.10
    node: 6.17.1
    maven: 3.6.3

Final Build Summary:

[INFO] Reactor Summary:
[INFO] 
[INFO] Metron 0.7.2 ....................................... SUCCESS [ 13.246 s]
[INFO] metron-stellar 0.7.2 ............................... SUCCESS [ 10.567 s]
[INFO] stellar-common 0.7.2 ............................... SUCCESS [02:49 min]
[INFO] metron-analytics 0.7.2 ............................. SUCCESS [  2.595 s]
[INFO] metron-maas-common 0.7.2 ........................... SUCCESS [  3.928 s]
[INFO] metron-platform 0.7.2 .............................. SUCCESS [  0.203 s]
[INFO] metron-zookeeper 0.7.2 ............................. SUCCESS [  0.695 s]
[INFO] metron-test-utilities 0.7.2 ........................ SUCCESS [ 34.358 s]
[INFO] metron-integration-test 0.7.2 ...................... SUCCESS [ 38.603 s]
[INFO] metron-maas-service 0.7.2 .......................... SUCCESS [  5.446 s]
[INFO] metron-common 0.7.2 ................................ SUCCESS [ 42.033 s]
[INFO] metron-statistics 0.7.2 ............................ SUCCESS [ 11.303 s]
[INFO] metron-common-streaming 0.7.2 ...................... SUCCESS [  0.138 s]
[INFO] metron-common-storm 0.7.2 .......................... SUCCESS [  1.258 s]
[INFO] metron-hbase 0.7.2 ................................. SUCCESS [  0.132 s]
[INFO] metron-hbase-common 0.7.2 .......................... SUCCESS [ 27.949 s]
[INFO] metron-enrichment 0.7.2 ............................ SUCCESS [  0.171 s]
[INFO] metron-enrichment-common 0.7.2 ..................... SUCCESS [ 20.860 s]
[INFO] metron-writer 0.7.2 ................................ SUCCESS [  0.132 s]
[INFO] metron-writer-common 0.7.2 ......................... SUCCESS [  7.072 s]
[INFO] metron-writer-storm 0.7.2 .......................... SUCCESS [ 18.700 s]
[INFO] metron-storm-kafka-override 0.7.2 .................. SUCCESS [  9.467 s]
[INFO] metron-storm-kafka 0.7.2 ........................... SUCCESS [  0.640 s]
[INFO] metron-profiler-common 0.7.2 ....................... SUCCESS [  1.150 s]
[INFO] metron-profiler-client 0.7.2 ....................... SUCCESS [ 15.490 s]
[INFO] metron-profiler-storm 0.7.2 ........................ SUCCESS [ 21.331 s]
[INFO] metron-profiler-spark 0.7.2 ........................ SUCCESS [02:31 min]
[INFO] metron-profiler-repl 0.7.2 ......................... SUCCESS [  7.415 s]
[INFO] metron-hbase-client 0.7.2 .......................... SUCCESS [ 12.459 s]
[INFO] metron-enrichment-storm 0.7.2 ...................... SUCCESS [ 20.594 s]
[INFO] metron-indexing 0.7.2 .............................. SUCCESS [  0.151 s]
[INFO] metron-indexing-common 0.7.2 ....................... SUCCESS [ 11.446 s]
[INFO] metron-solr 0.7.2 .................................. SUCCESS [  0.105 s]
[INFO] metron-solr-common 0.7.2 ........................... SUCCESS [01:18 min]
[INFO] metron-indexing-storm 0.7.2 ........................ SUCCESS [  0.621 s]
[INFO] metron-solr-storm 0.7.2 ............................ SUCCESS [ 18.554 s]
[INFO] metron-job 0.7.2 ................................... SUCCESS [  0.604 s]
[INFO] metron-pcap 0.7.2 .................................. SUCCESS [  4.397 s]
[INFO] metron-pcap-backend 0.7.2 .......................... SUCCESS [ 21.134 s]
[INFO] metron-data-management 0.7.2 ....................... SUCCESS [ 37.270 s]
[INFO] metron-parsing 0.7.2 ............................... SUCCESS [  0.115 s]
[INFO] metron-parsers-common 0.7.2 ........................ SUCCESS [ 29.593 s]
[INFO] metron-parsers 0.7.2 ............................... SUCCESS [ 22.274 s]
[INFO] metron-management 0.7.2 ............................ SUCCESS [  6.627 s]
[INFO] elasticsearch-shaded 0.7.2 ......................... SUCCESS [ 51.804 s]
[INFO] metron-elasticsearch 0.7.2 ......................... SUCCESS [  0.095 s]
[INFO] metron-elasticsearch-common 0.7.2 .................. SUCCESS [  9.219 s]
[INFO] metron-elasticsearch-storm 0.7.2 ................... SUCCESS [ 20.834 s]
[INFO] metron-parsing-storm 0.7.2 ......................... SUCCESS [ 22.158 s]
[INFO] metron-hbase-server 0.7.2 .......................... SUCCESS [ 10.565 s]
[INFO] metron-deployment 0.7.2 ............................ SUCCESS [  0.249 s]
[INFO] Elasticsearch Ambari Management Pack 0.7.2.0 ....... SUCCESS [  0.174 s]
[INFO] Metron Ambari Management Pack 0.7.2.0 .............. SUCCESS [  1.012 s]
[INFO] metron-contrib 0.7.2 ............................... SUCCESS [  0.090 s]
[INFO] metron-docker 0.7.2 ................................ SUCCESS [  0.641 s]
[INFO] metron-performance 0.7.2 ........................... SUCCESS [  1.571 s]
[INFO] metron-interface 0.7.2 ............................. SUCCESS [  0.129 s]
[INFO] metron-config 0.7.2 ................................ SUCCESS [02:22 min]
[INFO] metron-alerts 0.7.2 ................................ SUCCESS [03:11 min]
[INFO] metron-rest-client 0.7.2 ........................... SUCCESS [  7.283 s]
[INFO] metron-rest 0.7.2 .................................. SUCCESS [01:53 min]
[INFO] site-book 0.7.2 .................................... SUCCESS [  4.580 s]
[INFO] 3rd party Functions (just for tests) 0.7.2 ......... SUCCESS [  0.933 s]
[INFO] stellar-zeppelin 0.7.2 ............................. SUCCESS [ 29.864 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24:59 min
[INFO] Finished at: 2020-05-28T12:58:08Z
[INFO] ------------------------------------------------------------------------

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

Thanks,

Steven @ DFHZ

Re: Metron Build fails on metron config - single node vagrant VM

HarshR — Mon, 01 Jun 2020 13:42:14 GMT

I would also like to add that for some reason when I did this with CentOS with GNOME desktop, this fails. But if I run a minimal install, this works. I'm not sure if this is because of some python libraries being downloaded for GNOME..

Thank you for getting me to the next steps!

Re: Metron Build fails on metron config - single node vagrant VM

stevenmatison — Mon, 01 Jun 2020 13:46:39 GMT

@HarshR No problem.....

Centos with a Desktop not working right, NOoooooo say it aint so....