[Apache Metron] Ansible Failed To Complete Successfully

[#1]

Hi,

my platform-info.sh gives me this:

Metron 0.6.0
--
* master
--
commit 62310743a0c1d4a27db47a89cf79496d594c8bcf
Author: justinleet <justinjleet@gmail.com>
Date:   Tue Sep 4 19:53:37 2018 -0400
    METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183
--
--
ansible 2.5.0
  config file = None
  configured module search path = [u'/home/akaticentos6/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
--
Vagrant 2.1.4
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.15
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /opt/maven
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_181-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-754.el6.x86_64", arch: "amd64", family: "unix"
--
Docker version 1.7.1, build 786b29d/1.7.1
--
node
v0.10.48
--
npm
1.3.6
--
g++ (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--
Compiler is C++11 compliant
--
Linux localhost.localdomain 2.6.32-754.el6.x86_64 #1 SMP Tue Jun 19 21:26:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 9075.27 MB
Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Processor Speed: 3392.420 MHz
Total Physical Processors: 2
Total cores: 2
Disk information:
/dev/mapper/vg_akaticentos6-lv_root
/dev/sda1             477M   35M  417M   8% /boot
/dev/sr0               71M   71M     0 100% /media/VMware Tools
This CPU appears to support virtualization

It seems all pre-requisites are fulfilled.

But when I run vagrant provision for metron-deployment/development/centos6, it shows this:

TASK [metron-builder : Build Metron RPM Packages] ******************************
failed: [node1 -> localhost] (item=mvn package -DskipTests -Pbuild-rpms -T 2C) => {"changed": true, "cmd": "mvn package -DskipTests -Pbuild-rpms -T 2C", "delta": "0:00:22.816706", "end": "2018-09-06 16:36:28.706172", "item": "mvn package -DskipTests -Pbuild-rpms -T 2C", "msg": "non-zero return code", "rc": 1, "start": "2018-09-06 16:36:05.889466", "stderr": "Post http:///var/run/docker.sock/v1.19/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=... dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?", "stderr_lines": ["Post http:///var/run/docker.sock/v1.19/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=... dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?"], "stdout": "[INFO] Scanning for projects...\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Build Order:\n[INFO] \n[INFO] metron-deployment\n[INFO] metron-rpm\n[INFO] \n[INFO] Using the MultiThreadedBuilder implementation with a thread count of 4\n[INFO]                                                                         \n[INFO] ------------------------------------------------------------------------\n[INFO] Building metron-deployment 0.6.0\n[INFO] ------------------------------------------------------------------------\n[INFO] \n[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-deployment ---\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-deployment ---\n[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/target/jacoco.exec\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-deployment ---\n[INFO] Skipping JaCoCo execution due to missing execution data file.\n[INFO]                                                                         \n[INFO] ------------------------------------------------------------------------\n[INFO] Building metron-rpm 0.6.0\n[INFO] ------------------------------------------------------------------------\n[INFO] \n[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-rpm ---\n[INFO] \n[INFO] --- maven-antrun-plugin:1.1:run (default) @ metron-rpm ---\n[INFO] Executing tasks\n     [echo] ******** Displaying value of property ********\n     [echo] ../../../..\n[INFO] Executed tasks\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-rpm ---\n[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/packaging/docker/rpm-docker/target/jacoco.exec\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-rpm ---\n[INFO] Skipping JaCoCo execution due to missing execution data file.\n[INFO] \n[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-rpm-sources) @ metron-rpm ---\n[INFO] Using 'UTF-8' encoding to copy filtered resources.\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] \n[INFO] --- exec-maven-plugin:1.5.0:exec (docker-build) @ metron-rpm ---\nSending build context to Docker daemon \r\n[ERROR] Command execution failed.\norg.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)\n\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)\n\tat org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)\n\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:764)\n\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:711)\n\tat org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:289)\n\tat org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)\n\tat org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)\n\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)\n\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Summary:\n[INFO] \n[INFO] metron-deployment .................................. SUCCESS [  1.499 s]\n[INFO] metron-rpm ......................................... FAILURE [  8.590 s]\n[INFO] ------------------------------------------------------------------------\n[INFO] BUILD FAILURE\n[INFO] ------------------------------------------------------------------------\n[INFO] Total time: 10.252 s (Wall Clock)\n[INFO] Finished at: 2018-09-06T16:36:17+08:00\n[INFO] Final Memory: 15M/244M\n[INFO] ------------------------------------------------------------------------\n[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (docker-build) on project metron-rpm: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]\n[ERROR] \n[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.\n[ERROR] Re-run Maven using the -X switch to enable full debug logging.\n[ERROR] \n[ERROR] For more information about the errors and possible solutions, please read the following articles:\n[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException\n[ERROR] \n[ERROR] After correcting the problems, you can resume the build with the command\n[ERROR]   mvn <goals> -rf :metron-rpm", "stdout_lines": ["[INFO] Scanning for projects...", "[INFO] ------------------------------------------------------------------------", "[INFO] Reactor Build Order:", "[INFO] ", "[INFO] metron-deployment", "[INFO] metron-rpm", "[INFO] ", "[INFO] Using the MultiThreadedBuilder implementation with a thread count of 4", "[INFO]                                                                         ", "[INFO] ------------------------------------------------------------------------", "[INFO] Building metron-deployment 0.6.0", "[INFO] ------------------------------------------------------------------------", "[INFO] ", "[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-deployment ---", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-deployment ---", "[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/target/jacoco.exec", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-deployment ---", "[INFO] Skipping JaCoCo execution due to missing execution data file.", "[INFO]                                                                         ", "[INFO] ------------------------------------------------------------------------", "[INFO] Building metron-rpm 0.6.0", "[INFO] ------------------------------------------------------------------------", "[INFO] ", "[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-rpm ---", "[INFO] ", "[INFO] --- maven-antrun-plugin:1.1:run (default) @ metron-rpm ---", "[INFO] Executing tasks", "     [echo] ******** Displaying value of property ********", "     [echo] ../../../..", "[INFO] Executed tasks", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-rpm ---", "[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/packaging/docker/rpm-docker/target/jacoco.exec", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-rpm ---", "[INFO] Skipping JaCoCo execution due to missing execution data file.", "[INFO] ", "[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-rpm-sources) @ metron-rpm ---", "[INFO] Using 'UTF-8' encoding to copy filtered resources.", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] ", "[INFO] --- exec-maven-plugin:1.5.0:exec (docker-build) @ metron-rpm ---", "Sending build context to Docker daemon ", "[ERROR] Command execution failed.", "org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)", "\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)", "\tat org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)", "\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:764)", "\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:711)", "\tat org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:289)", "\tat org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)", "\tat org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)", "\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)", "\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)", "\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)", "\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)", "\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)", "\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)", "\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)", "\tat java.lang.Thread.run(Thread.java:748)", "[INFO] ------------------------------------------------------------------------", "[INFO] Reactor Summary:", "[INFO] ", "[INFO] metron-deployment .................................. SUCCESS [  1.499 s]", "[INFO] metron-rpm ......................................... FAILURE [  8.590 s]", "[INFO] ------------------------------------------------------------------------", "[INFO] BUILD FAILURE", "[INFO] ------------------------------------------------------------------------", "[INFO] Total time: 10.252 s (Wall Clock)", "[INFO] Finished at: 2018-09-06T16:36:17+08:00", "[INFO] Final Memory: 15M/244M", "[INFO] ------------------------------------------------------------------------", "[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (docker-build) on project metron-rpm: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]", "[ERROR] ", "[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.", "[ERROR] Re-run Maven using the -X switch to enable full debug logging.", "[ERROR] ", "[ERROR] For more information about the errors and possible solutions, please read the following articles:", "[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException", "[ERROR] ", "[ERROR] After correcting the problems, you can resume the build with the command", "[ERROR]   mvn <goals> -rf :metron-rpm"]}
NO MORE HOSTS LEFT *************************************************************
    to retry, use: --limit @/home/.../metron/metron-deployment/development/centos6/ansible/playbook.retry
PLAY RECAP *********************************************************************
node1                      : ok=9    changed=3    unreachable=0    failed=1   
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

What should I check?

[#2 · Accepted solution]

Hi @Nor Liana Kamaruzzaman ,

I compared with my environment and see a couple of differences...

Here's my output of the platform-info script

	➜  metron git:(master) ./metron-deployment/scripts/platform-info.sh
Metron 0.5.1
--
* master
--
commit d32bd50d43aae87af9ec12d2daea83b2f4eca342 (HEAD -> master, origin/master, origin/HEAD)
Author: nickwallen <nick@nickallen.org>
Date:   Wed Aug 29 14:55:58 2018 -0400
    METRON-1751 Storm Profiler dies when consuming null message (nickwallen) closes apache/metron#1176
--
--
ansible 2.2.2.0
  config file =
  configured module search path = Default w/o overrides
--
Vagrant 2.0.2
--
vagrant-hostmanager (1.8.9)
--
Python 2.7.15
--
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T00:03:14+05:30)
Maven home: /usr/local/Cellar/maven/3.5.4/libexec
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
--
Docker version 17.12.0-ce, build c97c6d6
--
node
v9.11.1
--
npm
5.6.0
--
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
--
Compiler is C++11 compliant
--
Darwin HW10098-2.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
--
Total System Memory = 16384 MB
Processor Model: Intel(R) Core(TM) i7-4770HQ CPU
Processor Speed: 2.20GHz
Total Physical Processors: 4
Total cores: 4
Disk information:
/dev/disk1s1   466Gi  187Gi  276Gi    41% 1849664 9223372036852926143    0%   /
/dev/disk1s4   466Gi  2.0Gi  276Gi     1%       2 9223372036854775805    0%   /private/var/vm
This CPU appears to support virtualization
	

From your output, I can see that the version of Docker and Node/npm are dated. Can you upgrade Docker to a recent version and node to ver 9.x and give it a try again?

[#3]

Failing in metron-rpm usually means that docker is not running

[#4]

@Otto Fowler: when I run 'service docker status' it says:

docker (pid 7232) is running...

I assume it is running? This is prior to running 'vagrant up'

[#5]

Yes, it means Docker is running. However I see that it is an old version.

Can you try with a more recent version of Docker and node ?

[#6]

hi @asubramanian

Would that require CentOS 7? Just to note, I am using CentOS 6.

update:Couldn't.

"systemd-units is needed by docker-ce-18.06.1.ce-3.el7.x86_64"

Latest one requires it. CentOS 6 does not support it. Doesn't seem to have any option to upgrade CentOS 6 to CentOS 7/

[#7]

Can you build manually? Check the readme in the rpm project and try to build it, you will get the std out of the rpm build and there will be more info

[#8]

@asubramanian: I tried to update npm, but it returned this:

npm ERR! Error: CERT_UNTRUSTED
npm ERR!     at SecurePair.<anonymous> (tls.js:1430:32)
npm ERR!     at SecurePair.emit (events.js:92:17)
npm ERR!     at SecurePair.maybeInitFinished (tls.js:1029:10)
npm ERR!     at CleartextStream.read [as _read] (tls.js:521:13)
npm ERR!     at CleartextStream.Readable.read (_stream_readable.js:341:10)
npm ERR!     at EncryptedStream.write [as _write] (tls.js:418:25)
npm ERR!     at doWrite (_stream_writable.js:226:10)
npm ERR!     at writeOrBuffer (_stream_writable.js:216:5)
npm ERR!     at EncryptedStream.Writable.write (_stream_writable.js:183:11)
npm ERR!     at write (_stream_readable.js:602:24)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Linux 2.6.32-754.el6.x86_64
npm ERR! command "node" "/usr/bin/npm" "install"
npm ERR! cwd /usr/lib/node_modules/npm
npm ERR! node -v v0.10.48
npm ERR! npm -v 1.3.6
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /usr/lib/node_modules/npm/npm-debug.log
npm ERR! not ok code 0

I had run update, but the error still appear after update.

[#9]

Looks like the node and npm versions are still old. You can grab the ver 9.11.2 version of nodejs from here - https://nodejs.org/download/release/v9.11.2/

[#10]

Hi all, so I am able to update my node and npm, but not docker:

Metron 0.6.0
--
* master
--
commit 62310743a0c1d4a27db47a89cf79496d594c8bcf
Author: justinleet <justinjleet@gmail.com>
Date:   Tue Sep 4 19:53:37 2018 -0400

    METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183
--
 metron-deployment/development/centos6/Vagrantfile |  168 ++++++++------------
 1 files changed, 67 insertions(+), 101 deletions(-)
--
ansible 2.5.0
  config file = None
  configured module search path = [u'/home/akaticentos6/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
--
Vagrant 2.1.4
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.15
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /opt/maven
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_181-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-754.el6.x86_64", arch: "amd64", family: "unix"
--
Docker version 1.7.1, build 786b29d/1.7.1
--
node
v9.11.2
--
npm
6.4.1
--
g++ (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

--
Compiler is C++11 compliant
--
Linux localhost.localdomain 2.6.32-754.el6.x86_64 #1 SMP Tue Jun 19 21:26:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 9075.27 MB
Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Processor Speed: 3392.349 MHz
Total Physical Processors: 2
Total cores: 2
Disk information:
/dev/mapper/vg_akaticentos6-lv_root
/dev/sda1             477M   67M  385M  15% /boot
/dev/sr0               71M   71M     0 100% /media/VMware Tools
This CPU appears to support virtualization

When I try to run vagrant up in metron/metron-deployment/development/centos6 , this show:

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'centos/6'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'centos/6' is up to date...
==> default: Setting the name of the VM: centos6_default_1537433543787_6601
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: No guest additions were detected on the base box for this VM! Guest
    default: additions are required for forwarded ports, shared folders, host only
    default: networking, and more. If SSH fails on this machine, please install
    default: the guest additions and repackage the box to continue.
    default: 
    default: This is not an error message; everything may continue to work properly,
    default: in which case you may ignore this message.
==> default: Rsyncing folder: /home/akaticentos6/metron/metron-deployment/development/centos6/ => /vagrant

Also, I notice that in my metron-deployment, there's no vagrant folder:

cd metron-deployment
$ dir
amazon-ec2  ansible.cfg  Kerberos-ambari-setup.md  other-examples  pom.xml      README.md
ansible     development  Kerberos-manual-setup.md  packaging       readme-images  scripts

but I saw in other tutorials, there supposed to be one. Is this normal?

[#11]

Hi @Nor Liana Kamaruzzaman , the vagrant folder not being present seems to be okay. Did the vagrant up command proceed further after upgrading the node and npm?

[#12]

Hi @asubramanian,

It stopped here, what should I do next?

==> default: Rsyncing folder: /home/akaticentos6/metron/metron-deployment/development/centos6/ => /vagrant

[#13]

Hm.. that is strange. I see from your output of platform-info.sh that you have the right version of the vagrant hostmanager plugin (ver 1.8.9). The one difference I see is that your machine is named as 'default', while it should show as 'node1'. See sample output from my box. Observe the logs..

➜  ~ vagrant up
 Running with ansible-skip-tags: ["sensors"]
Bringing machine 'node1' up with 'virtualbox' provider...
==> node1: Importing base box 'centos/6'...
==> node1: Matching MAC address for NAT networking...
==> node1: Checking if box 'centos/6' is up to date...
==> node1: Setting the name of the VM: centos6_node1_1537451480618_72587
==> node1: Clearing any previously set network interfaces...
==> node1: Preparing network interfaces based on configuration...
    node1: Adapter 1: nat
    node1: Adapter 2: hostonly
==> node1: Forwarding ports...
    node1: 22 (guest) => 2222 (host) (adapter 1)
==> node1: Running 'pre-boot' VM customizations...
==> node1: Booting VM...
==> node1: Waiting for machine to boot. This may take a few minutes...
    node1: SSH address: 127.0.0.1:2222
    node1: SSH username: vagrant
    node1: SSH auth method: private key
    node1:
    node1: Vagrant insecure key detected. Vagrant will automatically replace
    node1: this with a newly generated keypair for better security.
    node1:
    node1: Inserting generated public key within guest...
    node1: Removing insecure key from the guest if it's present...
    node1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> node1: Machine booted and ready!
==> node1: Checking for guest additions in VM...
    node1: No guest additions were detected on the base box for this VM! Guest
    node1: additions are required for forwarded ports, shared folders, host only
    node1: networking, and more. If SSH fails on this machine, please install
    node1: the guest additions and repackage the box to continue.
    node1:
    node1: This is not an error message; everything may continue to work properly,
    node1: in which case you may ignore this message.
==> node1: Setting hostname...
==> node1: Configuring and enabling network interfaces...
    node1: SSH address: 127.0.0.1:2222
    node1: SSH username: vagrant
    node1: SSH auth method: private key
==> node1: Rsyncing folder: /Users/asubramanian/Desktop/Metron/anand-metron-fork/shane-cypress-test/metron/metron-deployment/development/centos6/ => /vagrant
==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...
Password:
==> node1: Running provisioner: ansible...
    node1: Running ansible-playbook...
PLAY [all] *********************************************************************
<br>

Can you confirm that you are running the vagrant up command from the following folder location:

metron/metron-deployment/development/centos6

Also, as suggested by @Otto Fowler, can you try to compile outside of the full-dev deployment and see if that works?

You can do this like follows and look for any failures in the output.:

cd <root-level-metron-folder>
mvn clean package

I would also like to look at your Vagrantfile at the following location. Can you attach it?

metron/metron-deployment/development/centos6/Vagrantfile

[#14]

Hi @asubramanian,

I re-create the Vagrantfile using 'rm -r Vagrantfile' and 'vagrant init centos/6' And then I run vagrant up, and now I got this: (had to put in pastebin as it exceeds char limit) https://pastebin.com/x99i5zpK

Now I'll try clean packages

[#15]

@asubramanian

This is the result when I run 'mvn -e package' : https://pastebin.com/3UDfH6GT

[#16]

Hey @Nor Liana Kamaruzzaman, this definitely looks to me like an issue with the Docker version. This error message especially:

"Sending build context to Docker daemon ", "[ERROR] Command execution failed.", "org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)", "\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)", "\tat 

Would you be able to bring up a CentOS 7 environment afresh? It looks like the Docker pre-requisites asks for CentOS7. Here is an excerpt from their docs:

https://docs.docker.com/install/linux/docker-ce/centos/#prerequisites

OS requirements

To install Docker CE, you need a maintained version of CentOS 7. Archived versions aren’t supported or tested.

[#17]

Hey @asubramanian,

Thank you for waiting.I will try 🙂

Before I proceed, which one should I follow to try on CentOS 7?
Dev VM install for CentOS or Metron 0.4.1 with HDP 2.5 bare-metal install on Centos 7 with MariaDB for Metron REST. My system is only 12GB RAM and dual processor thought.

[#18]

Hey @Nor Liana Kamaruzzaman , cool! I would suggest that you try with the Dev VM install for CentOS link, and with the latest 'master' branch. It should be pretty straight forward once you have the pre-requisites setup. Note that the latest master requires ansible version to be 2.4.0+ (see latest instructions here).

[#19]

Hi @asubramanian,

Recently I tried with CentOS 7, but ran into a problem:

Message from syslogd@localhost at Oct  8 13:32:47 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [nautilus-deskto:2881]

Message from syslogd@localhost at Oct  8 13:33:32 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#0 stuck for 37s! [kworker/0:1:13622]
fatal: [node1]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Connection timed out during banner exchange\r\n", "unreachable": true}
    to retry, use: --limit @/home/<user>/metron/metron-deployment/development/centos6/ansible/playbook.retry

PLAY RECAP *********************************************************************
node1                      : ok=23   changed=14   unreachable=1    failed=0   

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Below is the result of platform-info.sh:

Metron 0.6.0
--
* Metron_0.6.0
--
commit 62310743a0c1d4a27db47a89cf79496d594c8bcf
Author: justinleet <justinjleet@gmail.com>
Date:   Tue Sep 4 19:53:37 2018 -0400

    METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183
--
--
ansible 2.5.0
  config file = /home/<user>/metron/metron-deployment/ansible.cfg
  configured module search path = [u'/home/<user>/metron/metron-deployment/extra_modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
--
Vagrant 2.1.5
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.5
--
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00)
Maven home: /usr/local/src/apache-maven
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-862.14.4.el7.x86_64", arch: "amd64", family: "unix"
--
Docker version 18.09.0-beta3, build 7718f80
--
node
scripts/platform-info.sh: line 77: node: command not found
--
npm
scripts/platform-info.sh: line 82: npm: command not found
--
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

--
Compiler is C++11 compliant
--
Linux localhost.localdomain 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 9670.97 MB
Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Processor Speed: 3392.460 MHz
Total Physical Processors: 2
Total cores: 2
Disk information:
/dev/mapper/centos-root   45G   20G   25G  44% /
/dev/sda1               1014M  221M  794M  22% /boot
This CPU appears to support virtualization

Please advice...

[#20]

Hi @Nor Liana Kamaruzzaman,

Can you try this solution here for the timeout issue with the vagrant VM?

Also, from the platform-info script, I can see that node JS installation is missing. Can you fix that too and try again?

node
scripts/platform-info.sh: line 77: node: command not found
--
npm
scripts/platform-info.sh: line 82: npm: command not found

[#21]

hi @asubramanian,

I've installed node.js. :

Metron 0.6.0
--
* Metron_0.6.0
--
commit 62310743a0c1d4a27db47a89cf79496d594c8bcf
Author: justinleet <justinjleet@gmail.com>
Date:   Tue Sep 4 19:53:37 2018 -0400
    METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183
--
 metron-deployment/development/centos6/ansible.cfg          |  3 +++
 metron-deployment/development/centos6/ansible/playbook.yml | 12 ++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)
--
ansible 2.5.0
  config file = None
  configured module search path = [u'/home/<user>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
--
Vagrant 2.1.5
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.5
--
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00)
Maven home: /usr/local/src/apache-maven
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-862.14.4.el7.x86_64", arch: "amd64", family: "unix"
--
Docker version 18.09.0-beta3, build 7718f80
--
node
v6.14.3
--
npm
3.10.10
--
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--
Compiler is C++11 compliant
--
Linux localhost.localdomain 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 9670.98 MB
Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Processor Speed: 3392.409 MHz
Total Physical Processors: 2
Total cores: 2
Disk information:
/dev/mapper/centos-root   45G   18G   27G  41% /
/dev/sda1               1014M  221M  794M  22% /boot
This CPU appears to support virtualization
<br>

This is the error I got, (6000 char exceeded, so I put it on pastebin):Metron Fourth Error Issue