Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 09-06-2018 12:27 PM
Hi,
my platform-info.sh gives me this:
Metron 0.6.0 -- * master -- commit 62310743a0c1d4a27db47a89cf79496d594c8bcf Author: justinleet <[email protected]> Date: Tue Sep 4 19:53:37 2018 -0400 METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183 -- -- ansible 2.5.0 config file = None configured module search path = [u'/home/akaticentos6/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.6/site-packages/ansible executable location = /usr/bin/ansible python version = 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] -- Vagrant 2.1.4 -- vagrant-hostmanager (1.8.9, global) -- Python 2.7.15 -- Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) Maven home: /opt/maven Java version: 1.8.0_181, vendor: Oracle Corporation Java home: /usr/java/jdk1.8.0_181-amd64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.32-754.el6.x86_64", arch: "amd64", family: "unix" -- Docker version 1.7.1, build 786b29d/1.7.1 -- node v0.10.48 -- npm 1.3.6 -- g++ (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15) Copyright (C) 2013 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -- Compiler is C++11 compliant -- Linux localhost.localdomain 2.6.32-754.el6.x86_64 #1 SMP Tue Jun 19 21:26:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux -- Total System Memory = 9075.27 MB Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Processor Speed: 3392.420 MHz Total Physical Processors: 2 Total cores: 2 Disk information: /dev/mapper/vg_akaticentos6-lv_root /dev/sda1 477M 35M 417M 8% /boot /dev/sr0 71M 71M 0 100% /media/VMware Tools This CPU appears to support virtualization
It seems all pre-requisites are fulfilled.
But when I run vagrant provision for metron-deployment/development/centos6, it shows this:
TASK [metron-builder : Build Metron RPM Packages] ******************************
failed: [node1 -> localhost] (item=mvn package -DskipTests -Pbuild-rpms -T 2C) => {"changed": true, "cmd": "mvn package -DskipTests -Pbuild-rpms -T 2C", "delta": "0:00:22.816706", "end": "2018-09-06 16:36:28.706172", "item": "mvn package -DskipTests -Pbuild-rpms -T 2C", "msg": "non-zero return code", "rc": 1, "start": "2018-09-06 16:36:05.889466", "stderr": "Post http:///var/run/docker.sock/v1.19/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=... dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?", "stderr_lines": ["Post http:///var/run/docker.sock/v1.19/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=... dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?"], "stdout": "[INFO] Scanning for projects...\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Build Order:\n[INFO] \n[INFO] metron-deployment\n[INFO] metron-rpm\n[INFO] \n[INFO] Using the MultiThreadedBuilder implementation with a thread count of 4\n[INFO] \n[INFO] ------------------------------------------------------------------------\n[INFO] Building metron-deployment 0.6.0\n[INFO] ------------------------------------------------------------------------\n[INFO] \n[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-deployment ---\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-deployment ---\n[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/target/jacoco.exec\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-deployment ---\n[INFO] Skipping JaCoCo execution due to missing execution data file.\n[INFO] \n[INFO] ------------------------------------------------------------------------\n[INFO] Building metron-rpm 0.6.0\n[INFO] ------------------------------------------------------------------------\n[INFO] \n[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-rpm ---\n[INFO] \n[INFO] --- maven-antrun-plugin:1.1:run (default) @ metron-rpm ---\n[INFO] Executing tasks\n [echo] ******** Displaying value of property ********\n [echo] ../../../..\n[INFO] Executed tasks\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-rpm ---\n[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/packaging/docker/rpm-docker/target/jacoco.exec\n[INFO] \n[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-rpm ---\n[INFO] Skipping JaCoCo execution due to missing execution data file.\n[INFO] \n[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-rpm-sources) @ metron-rpm ---\n[INFO] Using 'UTF-8' encoding to copy filtered resources.\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] Copying 1 resource\n[INFO] \n[INFO] --- exec-maven-plugin:1.5.0:exec (docker-build) @ metron-rpm ---\nSending build context to Docker daemon \r\n[ERROR] Command execution failed.\norg.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)\n\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)\n\tat org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)\n\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:764)\n\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:711)\n\tat org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:289)\n\tat org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)\n\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)\n\tat org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)\n\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)\n\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Summary:\n[INFO] \n[INFO] metron-deployment .................................. SUCCESS [ 1.499 s]\n[INFO] metron-rpm ......................................... FAILURE [ 8.590 s]\n[INFO] ------------------------------------------------------------------------\n[INFO] BUILD FAILURE\n[INFO] ------------------------------------------------------------------------\n[INFO] Total time: 10.252 s (Wall Clock)\n[INFO] Finished at: 2018-09-06T16:36:17+08:00\n[INFO] Final Memory: 15M/244M\n[INFO] ------------------------------------------------------------------------\n[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (docker-build) on project metron-rpm: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]\n[ERROR] \n[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.\n[ERROR] Re-run Maven using the -X switch to enable full debug logging.\n[ERROR] \n[ERROR] For more information about the errors and possible solutions, please read the following articles:\n[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException\n[ERROR] \n[ERROR] After correcting the problems, you can resume the build with the command\n[ERROR] mvn <goals> -rf :metron-rpm", "stdout_lines": ["[INFO] Scanning for projects...", "[INFO] ------------------------------------------------------------------------", "[INFO] Reactor Build Order:", "[INFO] ", "[INFO] metron-deployment", "[INFO] metron-rpm", "[INFO] ", "[INFO] Using the MultiThreadedBuilder implementation with a thread count of 4", "[INFO] ", "[INFO] ------------------------------------------------------------------------", "[INFO] Building metron-deployment 0.6.0", "[INFO] ------------------------------------------------------------------------", "[INFO] ", "[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-deployment ---", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-deployment ---", "[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/target/jacoco.exec", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-deployment ---", "[INFO] Skipping JaCoCo execution due to missing execution data file.", "[INFO] ", "[INFO] ------------------------------------------------------------------------", "[INFO] Building metron-rpm 0.6.0", "[INFO] ------------------------------------------------------------------------", "[INFO] ", "[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @ metron-rpm ---", "[INFO] ", "[INFO] --- maven-antrun-plugin:1.1:run (default) @ metron-rpm ---", "[INFO] Executing tasks", " [echo] ******** Displaying value of property ********", " [echo] ../../../..", "[INFO] Executed tasks", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default) @ metron-rpm ---", "[INFO] argLine set to -javaagent:/home/.../.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/.../metron/metron-deployment/packaging/docker/rpm-docker/target/jacoco.exec", "[INFO] ", "[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ metron-rpm ---", "[INFO] Skipping JaCoCo execution due to missing execution data file.", "[INFO] ", "[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-rpm-sources) @ metron-rpm ---", "[INFO] Using 'UTF-8' encoding to copy filtered resources.", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] Copying 1 resource", "[INFO] ", "[INFO] --- exec-maven-plugin:1.5.0:exec (docker-build) @ metron-rpm ---", "Sending build context to Docker daemon ", "[ERROR] Command execution failed.", "org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)", "\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)", "\tat org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)", "\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:764)", "\tat org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:711)", "\tat org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:289)", "\tat org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)", "\tat org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)", "\tat org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)", "\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)", "\tat org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)", "\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)", "\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)", "\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)", "\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)", "\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)", "\tat java.lang.Thread.run(Thread.java:748)", "[INFO] ------------------------------------------------------------------------", "[INFO] Reactor Summary:", "[INFO] ", "[INFO] metron-deployment .................................. SUCCESS [ 1.499 s]", "[INFO] metron-rpm ......................................... FAILURE [ 8.590 s]", "[INFO] ------------------------------------------------------------------------", "[INFO] BUILD FAILURE", "[INFO] ------------------------------------------------------------------------", "[INFO] Total time: 10.252 s (Wall Clock)", "[INFO] Finished at: 2018-09-06T16:36:17+08:00", "[INFO] Final Memory: 15M/244M", "[INFO] ------------------------------------------------------------------------", "[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (docker-build) on project metron-rpm: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]", "[ERROR] ", "[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.", "[ERROR] Re-run Maven using the -X switch to enable full debug logging.", "[ERROR] ", "[ERROR] For more information about the errors and possible solutions, please read the following articles:", "[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException", "[ERROR] ", "[ERROR] After correcting the problems, you can resume the build with the command", "[ERROR] mvn <goals> -rf :metron-rpm"]}
NO MORE HOSTS LEFT *************************************************************
to retry, use: --limit @/home/.../metron/metron-deployment/development/centos6/ansible/playbook.retry
PLAY RECAP *********************************************************************
node1 : ok=9 changed=3 unreachable=0 failed=1
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
What should I check?
Created 09-07-2018 06:12 AM
I compared with my environment and see a couple of differences...
Here's my output of the platform-info script
➜ metron git:(master) ./metron-deployment/scripts/platform-info.sh Metron 0.5.1 -- * master -- commit d32bd50d43aae87af9ec12d2daea83b2f4eca342 (HEAD -> master, origin/master, origin/HEAD) Author: nickwallen <[email protected]> Date: Wed Aug 29 14:55:58 2018 -0400 METRON-1751 Storm Profiler dies when consuming null message (nickwallen) closes apache/metron#1176 -- -- ansible 2.2.2.0 config file = configured module search path = Default w/o overrides -- Vagrant 2.0.2 -- vagrant-hostmanager (1.8.9) -- Python 2.7.15 -- Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T00:03:14+05:30) Maven home: /usr/local/Cellar/maven/3.5.4/libexec Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac" -- Docker version 17.12.0-ce, build c97c6d6 -- node v9.11.1 -- npm 5.6.0 -- Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.1.0 (clang-902.0.39.2) Target: x86_64-apple-darwin17.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin -- Compiler is C++11 compliant -- Darwin HW10098-2.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64 -- Total System Memory = 16384 MB Processor Model: Intel(R) Core(TM) i7-4770HQ CPU Processor Speed: 2.20GHz Total Physical Processors: 4 Total cores: 4 Disk information: /dev/disk1s1 466Gi 187Gi 276Gi 41% 1849664 9223372036852926143 0% / /dev/disk1s4 466Gi 2.0Gi 276Gi 1% 2 9223372036854775805 0% /private/var/vm This CPU appears to support virtualization
From your output, I can see that the version of Docker and Node/npm are dated. Can you upgrade Docker to a recent version and node to ver 9.x and give it a try again?
Created 10-09-2018 07:27 AM
hi @asubramanian,
I've installed node.js. :
Metron 0.6.0 -- * Metron_0.6.0 -- commit 62310743a0c1d4a27db47a89cf79496d594c8bcf Author: justinleet <[email protected]> Date: Tue Sep 4 19:53:37 2018 -0400 METRON-1764 Update version to 0.6.0 (justinleet) closes apache/metron#1183 -- metron-deployment/development/centos6/ansible.cfg | 3 +++ metron-deployment/development/centos6/ansible/playbook.yml | 12 ++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) -- ansible 2.5.0 config file = None configured module search path = [u'/home/<user>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] -- Vagrant 2.1.5 -- vagrant-hostmanager (1.8.9, global) -- Python 2.7.5 -- Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00) Maven home: /usr/local/src/apache-maven Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "3.10.0-862.14.4.el7.x86_64", arch: "amd64", family: "unix" -- Docker version 18.09.0-beta3, build 7718f80 -- node v6.14.3 -- npm 3.10.10 -- g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28) Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -- Compiler is C++11 compliant -- Linux localhost.localdomain 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux -- Total System Memory = 9670.98 MB Processor Model: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Processor Speed: 3392.409 MHz Total Physical Processors: 2 Total cores: 2 Disk information: /dev/mapper/centos-root 45G 18G 27G 41% / /dev/sda1 1014M 221M 794M 22% /boot This CPU appears to support virtualization <br>
This is the error I got, (6000 char exceeded, so I put it on pastebin):Metron Fourth Error Issue
Created 10-15-2018 05:09 AM
Hi @asubramanian,
I managed to troubleshoot the problem above by changing folder permission for the .npm and manually remove the _locks, then re-run vagrant up.
The thing is, when running vagrant provision, I noticed one of the task were building cluster Ambari. That shouldn't happen for single-VM, right? But it did.
The process ended due to several times CPU locked out/ overcommitment error, and then aborted. I have a feeling that this is because it is creating clusters instead of running on single-VM.
Created 10-15-2018 05:14 AM
Hi @Nor Liana Kamaruzzaman, the single node uses Ambari to deploy the cluster. So what you are seeing is correct.
I am suspecting if there is resource starvation because of which you are seeing the CPU locked out issues. Can you try to find a server with higher resources? Or upgrade the present server?
Created 10-17-2018 05:31 AM
Hey @Nor Liana Kamaruzzaman, this is great news! I am glad you were able to resolve the issues.
Would you mind accepting the answer that helped you the most?
Cheers,
Anand
Created 10-08-2018 05:41 AM
Hey @Nor Liana Kamaruzzaman ,
I am hoping you have got a chance to set up a different system with CentOS 7 and able to proceed further. In the context of this thread, would you mind marking the best answer if you feel the original question was answered? Thanks!
Created on 10-17-2018 01:04 AM - edited 08-18-2019 01:39 AM
Hi @asubramanian,
I've increased the host VM's RAM. It reduces the cpu lock out, and can finally run. 🙂 Thanks.