Created 11-26-2017 04:54 AM
Hi,
I am trying to deploy Apache Metron on a single node VM, but after vagrant up, when I run vagrant provision, it gives me errors on maven dependencies and ansible failed to setup successfully.
I have installed all packages defined for Metron deployment, Ansible 2.0.0.2, Vagrant 1.9.6, Maven 3.3.9, Virtual Box 5.1.30 JDK 1.8 and Python 2.7.5
Also, when I run the command "mvn clean package", metron-config error pops up and it doesnt complete its installation, followed by metron-rest, metron-rest client
Created 11-27-2017 10:05 AM
Hey @Gaurav Bapat, you need not run the mvn command every time. Just once should suffice. Also, if you have used the full-dev vagrant deployment, you need not have to do anything with Ambari UI (or the Step 3 screenshot) you have pasted.
So, let me paraphrase the steps for you so that you can give this a try once more, afresh. Let me know how it goes.
Assumption: Say your metron folder is located at: /user/gaurav/metron
* Run the following commands:
cd /user/gaurav/metron/metron-deployment/vagrant/full-dev-platform vagrant destroy -f
* The above destroy command would terminate and delete the existing running VM. You can also run 'VirtualBox' command to launch the application console, and can delete any other stray VMs that are not required.
* Run the following commands:
cd /user/gaurav/metron mvn clean compile -DskipTests
* The above command will take about 10 mins or so to complete, and should end with a "SUCCESS" message
* Run the following commands:
cd /user/gaurav/metron/metron-deployment/vagrant/full-dev-platform vagrant up
* The above command will take about 45 mins to over an hour to complete. I would recommend that you disable auto-sleep of the system in case it is enabled.
At the end of the step 2 above, you should be able to see a message saying that the deployment completed successfully.
Created 11-27-2017 05:49 AM
Hi @Gaurav Bapat, can you post the output of running the command metron-deployment/scripts/platform-info.sh
?
I am guessing this could be due to an older version of node and npm modules. Here is the output from my system, for which I am able to get everything working.
Metron 0.4.2 -- * master -- -- -- ansible 2.0.0.2 config file = configured module search path = Default w/o overrides -- Vagrant 1.9.4 -- Python 2.7.12 -- Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T22:11:47+05:30) Maven home: /usr/local/Cellar/maven/3.3.9/libexec Java version: 1.8.0_91, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac" -- Docker version 1.12.1, build 23cf638, experimental -- node v8.9.1 -- npm 5.5.1 -- Darwin HW12724.local 16.7.0 Darwin Kernel Version 16.7.0: Wed Oct 4 00:17:00 PDT 2017; root:xnu-3789.71.6~1/RELEASE_X86_64 x86_64 -- Total System Memory = 16384 MB Processor Model: Intel(R) Core(TM) i7-4870HQ CPU Processor Speed: 2.50GHz Total Physical Processors: 4 Total cores: 4 Disk information: /dev/disk1 465Gi 402Gi 63Gi 87% 2338554 4292628725 0% / /dev/disk3s1 96Mi 94Mi 1.7Mi 99% 12 4294967267 0% /Volumes/VirtualBox /dev/disk4s1 99Mi 97Mi 1.6Mi 99% 12 4294967267 0% /Volumes/VirtualBox 1 This CPU appears to support virtualization
Created 11-27-2017 06:51 AM
When I run vagrant provision, it gives me metron-config error, I ran mvn clean package command in metron-config folder but it fails to build metron-config and others successfully, please help or is the files in the tar corrupt?
@asubramanian
Created 11-27-2017 06:54 AM
Can you please run these steps and paste the output?
1. cd to the directory Metron top level directory (E.g. cd /home/user1/metron
)
2. Run ./metron-deployment/scripts/platform-info.sh
Created 11-27-2017 07:12 AM
Metron 0.4.1 fatal: Not a git repository (or any parent up to mount point /home) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). -- ansible 2.0.0.2 config file = configured module search path = Default w/o overrides -- Vagrant 1.9.6 -- Python 2.7.5 -- Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00) Maven home: /usr/local/maven/apache-maven-3.3.9 Java version: 1.8.0_151, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "3.10.0-693.el7.x86_64", arch: "amd64", family: "unix" -- Docker version 1.12.6, build 85d7426/1.12.6 -- node v6.12.0 -- npm 3.10.10 -- Linux metron.sequretek.com 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux -- Total System Memory = 9672.71 MB Processor Model: Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz Processor Speed: 3291.748 MHz Total Physical Processors: 2 Total cores: 2 Disk information: /dev/mapper/centos-root 50G 19G 31G 38% / /dev/sda1 1014M 179M 836M 18% /boot /dev/mapper/centos-home 65G 8.6G 56G 14% /home /dev/sr0 4.3G 4.3G 0 100% /run/media/gbapat/CentOS 7 x86_64 This CPU appears to support virtualization @asubramanian Thanks
Created 11-27-2017 07:18 AM
Sure.. you can follow these commands as per your OS - https://www.tecmint.com/install-nodejs-npm-in-centos-ubuntu/.
Please retry compiling Metron once you have upgraded the node. npm and let me know if it fixes the issue.
Created 11-27-2017 07:23 AM
It gives Skipping Ansible version error still
Created on 11-27-2017 07:28 AM - edited 08-17-2019 10:03 PM
after i run vagrant provision, this happensThanks @asubramanian
Created on 11-27-2017 07:38 AM - edited 08-17-2019 10:02 PM
@asubramanian Is it going good and what time will it take to complete?
Created on 11-27-2017 07:47 AM - edited 08-17-2019 10:02 PM
@asubramanian Why this no hosts matched? and I am so thankful that you helped me do this, Thanks alot!!
Created 11-27-2017 08:30 AM
Hey @Gaurav Bapat, you are most welcome! This is looking good. Depending on your the available system resources, the time to deploy will be anywhere between 45 mins to few hours.
Btw, would you mind marking the answer as accepted since the original issue is resolved?
Created on 11-27-2017 09:20 AM - edited 08-17-2019 10:02 PM
When I restarted my VM, same errors are popping up, like - mvn dependencies, do I need to enter mvn clean packages -Dskiptests everytime and which is the best directory for the same?
Also I am attaching step 3, please let me know how to setup SSH private key?
,
When I restarted the VM, it is again giving me same errors of mvn dependencies, do I need to again enter "mvn clean packages"? , also which is the best directory to enter this command?
I am attached a picture of Step 3 here, please let me know how to setup private SSH key.
@asubramanian Thanks
Created 11-27-2017 10:05 AM
Hey @Gaurav Bapat, you need not run the mvn command every time. Just once should suffice. Also, if you have used the full-dev vagrant deployment, you need not have to do anything with Ambari UI (or the Step 3 screenshot) you have pasted.
So, let me paraphrase the steps for you so that you can give this a try once more, afresh. Let me know how it goes.
Assumption: Say your metron folder is located at: /user/gaurav/metron
* Run the following commands:
cd /user/gaurav/metron/metron-deployment/vagrant/full-dev-platform vagrant destroy -f
* The above destroy command would terminate and delete the existing running VM. You can also run 'VirtualBox' command to launch the application console, and can delete any other stray VMs that are not required.
* Run the following commands:
cd /user/gaurav/metron mvn clean compile -DskipTests
* The above command will take about 10 mins or so to complete, and should end with a "SUCCESS" message
* Run the following commands:
cd /user/gaurav/metron/metron-deployment/vagrant/full-dev-platform vagrant up
* The above command will take about 45 mins to over an hour to complete. I would recommend that you disable auto-sleep of the system in case it is enabled.
At the end of the step 2 above, you should be able to see a message saying that the deployment completed successfully.
Created on 11-27-2017 10:23 AM - edited 08-17-2019 10:02 PM
When I re-run vagrant provision, it is giving me those errors back again 😞 @asubramanian
Created 11-27-2017 10:24 AM
Please dont re-run. Follow the 3 steps what I have mentioned above with the cleanup.
Created 11-27-2017 10:27 AM
Ahh.. wait @Gaurav Bapat, you also need to have Docker installed and running. In the above error message you can see that it says "Cannot connect to the Docker daemon".
Please install Docker (https://docs.docker.com/engine/installation) and ensure that it is running before you proceed with the deployment.
Created 11-27-2017 10:30 AM
Thank you so much, I am following your steps, but why am I getting npm dependency warnings or angular-cli ?? do i have to type any other command for it?
Created 11-27-2017 10:31 AM
Yeah, I have seen that too. I guess you can ignore them and move on, for now.
Created 11-27-2017 10:39 AM
@asubramanian How do I see the metron UI, node1:8888 or something else?
Created 11-27-2017 10:50 AM
Metron UI comprises the Kibana dashboard (node1:5000), service management (node1:2812), the Management UI (node1:4200) and the Alerts UI (node1:4201). Each of these cater to different things you would like to perform. The Metron site-book is an excellent place to know more about these - https://metron.apache.org/current-book/index.html. Give it a try.
Created 11-27-2017 10:57 AM
You are too good, and can I use Apache NiFi, Flume, Kafka and Spark??