Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Failing to Setup Apache Metron Dev Machine using Vagrant (Single Node on HDP 3.1 Branch)

Failing to Setup Apache Metron Dev Machine using Vagrant (Single Node on HDP 3.1 Branch)

New Contributor

I am setting up centos7, I already tried centos6 from other branches and got no luck in that, so I took a branch of metron hdp3.1 and followed the ReadMe, before starting I ran platform.sh and everything looked good..

On Vagrant Up it just fails on Metron_config with message

[node1]: FAILED! => {"changed": false, "msg": "Request failed with status FAILED"}

At this point I could open ambari but all services were down and each of them said the same thing, Failed to install service, Below are my logs

Spoiler

abidzaidi@abidzaidi:~/Source/metron/metron-deployment/development/centos7$ vagrant up
Running with ansible-skip-tags: ["sensors"]
Bringing machine 'node1' up with 'virtualbox' provider...
==> node1: Importing base box 'centos/7'...
==> node1: Matching MAC address for NAT networking...
==> node1: Checking if box 'centos/7' version '1905.1' is up to date...
==> node1: Setting the name of the VM: centos7_node1_1582905500842_94659
==> node1: Clearing any previously set network interfaces...
==> node1: Preparing network interfaces based on configuration...
node1: Adapter 1: nat
node1: Adapter 2: hostonly
==> node1: Forwarding ports...
node1: 22 (guest) => 2222 (host) (adapter 1)
==> node1: Running 'pre-boot' VM customizations...
==> node1: Booting VM...
==> node1: Waiting for machine to boot. This may take a few minutes...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
node1:
node1: Vagrant insecure key detected. Vagrant will automatically replace
node1: this with a newly generated keypair for better security.
node1:
node1: Inserting generated public key within guest...
node1: Removing insecure key from the guest if it's present...
node1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> node1: Machine booted and ready!
==> node1: Checking for guest additions in VM...
node1: No guest additions were detected on the base box for this VM! Guest
node1: additions are required for forwarded ports, shared folders, host only
node1: networking, and more. If SSH fails on this machine, please install
node1: the guest additions and repackage the box to continue.
node1:
node1: This is not an error message; everything may continue to work properly,
node1: in which case you may ignore this message.
==> node1: Setting hostname...
==> node1: Configuring and enabling network interfaces...
==> node1: Rsyncing folder: /home/abidzaidi/Source/metron/metron-deployment/development/centos7/ => /vagrant
==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...
==> node1: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.6.5).

Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode

node1: Running ansible-playbook...

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 20:58:57 +0500 (0:00:00.052) 0:00:00.052 *******
ok: [node1]

TASK [libselinux-python : Install libselinux-python] ***************************
Friday 28 February 2020 20:58:59 +0500 (0:00:02.009) 0:00:02.061 *******
ok: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:00 +0500 (0:00:00.890) 0:00:02.952 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/check-swap.yml for no1

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:00 +0500 (0:00:00.049) 0:00:03.002 *******
changed: [node1]

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.685) 0:00:03.687 *******
changed: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.625) 0:00:04.313 *******
skipping: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.047) 0:00:04.360 *******
ok: [node1] => {
"msg": "Swap is NOT enabled"
}

TASK [enable-swap : Checking for /swappy] **************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.096) 0:00:04.457 *******
ok: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.656) 0:00:05.113 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/setup-swap.yml for no1

TASK [enable-swap : Disable swap space] ****************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.071) 0:00:05.185 *******
changed: [node1]

TASK [enable-swap : Allocate 8092 mb for swap space] ***************************
Friday 28 February 2020 20:59:03 +0500 (0:00:00.781) 0:00:05.967 *******
changed: [node1]

TASK [enable-swap : Permissioning /swappy] *************************************
Friday 28 February 2020 20:59:22 +0500 (0:00:18.719) 0:00:24.687 *******
changed: [node1]

TASK [enable-swap : Setup swap space] ******************************************
Friday 28 February 2020 20:59:23 +0500 (0:00:00.764) 0:00:25.451 *******
changed: [node1]

TASK [enable-swap : Enable swap space] *****************************************
Friday 28 February 2020 20:59:30 +0500 (0:00:07.207) 0:00:32.659 *******
changed: [node1]

TASK [enable-swap : Enable swap on boot] ***************************************
Friday 28 February 2020 20:59:31 +0500 (0:00:01.208) 0:00:33.868 *******
changed: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.730) 0:00:34.599 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/check-swap.yml for no1

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.051) 0:00:34.650 *******
changed: [node1]

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.623) 0:00:35.274 *******
changed: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.620) 0:00:35.894 *******
ok: [node1] => {
"msg": "Swap is enabled with 8091 mb"
}

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.101) 0:00:35.996 *******
skipping: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.045) 0:00:36.042 *******
skipping: [node1]

TASK [enable-remote-ssh : Comment password auth no] ****************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.042) 0:00:36.084 *******
changed: [node1]

TASK [enable-remote-ssh : Uncomment password auth yes] *************************
Friday 28 February 2020 20:59:34 +0500 (0:00:00.611) 0:00:36.696 *******
changed: [node1]

TASK [enable-remote-ssh : Restart service sshd, in all cases] ******************
Friday 28 February 2020 20:59:34 +0500 (0:00:00.628) 0:00:37.324 *******
changed: [node1]

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 20:59:35 +0500 (0:00:00.788) 0:00:38.112 *******
ok: [node1]

TASK [metron-builder : Check distribution] *************************************
Friday 28 February 2020 20:59:36 +0500 (0:00:00.861) 0:00:38.974 *******
skipping: [node1]

TASK [metron-builder : Clean Metron] *******************************************
Friday 28 February 2020 20:59:36 +0500 (0:00:00.053) 0:00:39.027 *******
changed: [node1 -> localhost] => (item=mvn clean -P HDP-3.1)
changed: [node1 -> localhost] => (item=mvn clean -P mpack)
changed: [node1 -> localhost] => (item=mvn clean -P build-rpms)
changed: [node1 -> localhost] => (item=mvn clean -P build-debs)

TASK [metron-builder : Build Metron] *******************************************
Friday 28 February 2020 20:59:43 +0500 (0:00:06.876) 0:00:45.904 *******
changed: [node1 -> localhost] => (item=mvn package -DskipTests -T 2C -P HDP-3.1,mpack)

TASK [metron-builder : include_tasks] ******************************************
Friday 28 February 2020 21:07:59 +0500 (0:08:16.190) 0:09:02.095 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-builder/tasks/build-rpms.yml forode1

TASK [metron-builder : Build Metron RPM Packages] ******************************
Friday 28 February 2020 21:07:59 +0500 (0:00:00.067) 0:09:02.162 *******
^[[C^[[D
changed: [node1 -> localhost] => (item=mvn package -DskipTests -Pbuild-rpms -T 2C)

TASK [metron-builder : include_tasks] ******************************************
Friday 28 February 2020 21:22:01 +0500 (0:14:01.289) 0:23:03.451 *******
skipping: [node1]
[WARNING]: Could not match supplied host pattern, ignoring: ec2


PLAY [ec2] *********************************************************************
skipping: no hosts matched
[WARNING]: Could not match supplied host pattern, ignoring: packer


PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [search] ******************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:22:01 +0500 (0:00:00.061) 0:23:03.513 *******
ok: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:22:02 +0500 (0:00:01.858) 0:23:05.372 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.096) 0:23:05.468 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.820) 0:23:06.289 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.069) 0:23:06.358 *******
changed: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:23:53 +0500 (0:01:49.258) 0:24:55.617 *******
skipping: [node1]

TASK [solr : Create Solr user] *************************************************
Friday 28 February 2020 21:23:53 +0500 (0:00:00.051) 0:24:55.668 *******
changed: [node1]

TASK [solr : Download Solr] ****************************************************
Friday 28 February 2020 21:23:54 +0500 (0:00:00.833) 0:24:56.502 *******
changed: [node1]

TASK [solr : Extract Solr tarball] *********************************************
Friday 28 February 2020 21:28:35 +0500 (0:04:41.828) 0:29:38.330 *******
changed: [node1]

PLAY [ec2] *********************************************************************
skipping: no hosts matched

PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [ambari_*] ****************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:28:42 +0500 (0:00:06.200) 0:29:44.530 *******
ok: [node1]

TASK [libselinux-python : Install libselinux-python] ***************************
Friday 28 February 2020 21:28:43 +0500 (0:00:00.981) 0:29:45.512 *******
ok: [node1]

TASK [epel : Install EPEL repository] ******************************************
Friday 28 February 2020 21:28:44 +0500 (0:00:00.942) 0:29:46.454 *******
changed: [node1]

TASK [ntp : set_fact] **********************************************************
Friday 28 February 2020 21:28:46 +0500 (0:00:02.860) 0:29:49.315 *******
ok: [node1]

TASK [ntp : set_fact] **********************************************************
Friday 28 February 2020 21:28:47 +0500 (0:00:00.129) 0:29:49.444 *******
skipping: [node1]

TASK [ntp : Install ntp] *******************************************************
Friday 28 February 2020 21:28:47 +0500 (0:00:00.069) 0:29:49.514 *******
changed: [node1]

TASK [ntp : Ensure ntp is running and enabled] *********************************
Friday 28 February 2020 21:29:15 +0500 (0:00:28.419) 0:30:17.933 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:16 +0500 (0:00:00.876) 0:30:18.809 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/firewall-CentOS-7.y for node1

TASK [ambari_common : Stop firewalld on CentOS 7] ******************************
Friday 28 February 2020 21:29:16 +0500 (0:00:00.061) 0:30:18.871 *******
ok: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:17 +0500 (0:00:00.656) 0:30:19.527 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/hostname.yml for no1

TASK [ambari_common : Remove ipv4 'localhost.localdomain' from /etc/hosts] *****
Friday 28 February 2020 21:29:17 +0500 (0:00:00.058) 0:30:19.586 *******
changed: [node1]

TASK [ambari_common : Remove ipv6 'localhost.localdomain' from /etc/hosts] *****
Friday 28 February 2020 21:29:17 +0500 (0:00:00.621) 0:30:20.207 *******
changed: [node1]

TASK [ambari_common : Add localhost to /etc/hosts] *****************************
Friday 28 February 2020 21:29:18 +0500 (0:00:00.636) 0:30:20.844 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:19 +0500 (0:00:00.664) 0:30:21.509 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/ambari-repo-CentOS-yml for node1

TASK [ambari_common : Setup Ambari repo on CentOS] *****************************
Friday 28 February 2020 21:29:19 +0500 (0:00:00.061) 0:30:21.570 *******
changed: [node1]

TASK [ambari_common : Update package cache on CentOS] **************************
Friday 28 February 2020 21:29:20 +0500 (0:00:01.492) 0:30:23.062 *******
ok: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:30:26 +0500 (0:01:06.309) 0:31:29.372 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/nodejs-CentOS-7.ymlor node1

TASK [ambari_common : Install Node.js repo on CentOS 7] ************************
Friday 28 February 2020 21:30:27 +0500 (0:00:00.059) 0:31:29.432 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:30:51 +0500 (0:00:24.472) 0:31:53.905 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/logrotate.yml for ne1

TASK [ambari_common : Create Logrotate Script for Hadoop Services] *************
Friday 28 February 2020 21:30:51 +0500 (0:00:00.052) 0:31:53.958 *******
changed: [node1]

PLAY [ambari_master] ***********************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:30:52 +0500 (0:00:01.196) 0:31:55.155 *******
ok: [node1]

TASK [ambari_master : Install Ambari Server on CentOS] *************************
Friday 28 February 2020 21:30:53 +0500 (0:00:00.940) 0:31:56.096 *******
changed: [node1]

TASK [ambari_master : Install Ambari Server on Ubuntu] *************************
Friday 28 February 2020 21:42:33 +0500 (0:11:39.728) 0:43:35.824 *******
skipping: [node1]

TASK [ambari_master : Set Ambari Server Max Memory] ****************************
Friday 28 February 2020 21:42:33 +0500 (0:00:00.043) 0:43:35.867 *******
ok: [node1]

TASK [ambari_master : Allow 150 seconds of startup time for ambari server] *****
Friday 28 February 2020 21:42:34 +0500 (0:00:00.751) 0:43:36.619 *******
changed: [node1]

TASK [ambari_master : Setup Ambari Server] *************************************
Friday 28 February 2020 21:42:35 +0500 (0:00:01.260) 0:43:37.879 *******
changed: [node1]

TASK [ambari_master : Copy Metron MPack to Ambari Host] ************************
Friday 28 February 2020 21:48:23 +0500 (0:05:48.149) 0:49:26.028 *******
changed: [node1]

TASK [ambari_master : Install Metron MPack on Ambari Host] *********************
Friday 28 February 2020 21:48:25 +0500 (0:00:01.534) 0:49:27.562 *******
changed: [node1]

TASK [ambari_master : Copy Elasticsearch MPack to Ambari Host] *****************
Friday 28 February 2020 21:48:26 +0500 (0:00:01.142) 0:49:28.704 *******
changed: [node1]

TASK [ambari_master : Install Elasticsearch MPack on Ambari Host] **************
Friday 28 February 2020 21:48:27 +0500 (0:00:01.526) 0:49:30.231 *******
changed: [node1]

TASK [ambari_master : start ambari server] *************************************
Friday 28 February 2020 21:48:28 +0500 (0:00:01.106) 0:49:31.338 *******
changed: [node1]

PLAY [ambari_slave] ************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:48:48 +0500 (0:00:19.375) 0:49:50.713 *******
ok: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.886) 0:49:51.600 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.077) 0:49:51.677 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.562) 0:49:52.240 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.064) 0:49:52.304 *******
ok: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.775) 0:49:53.080 *******
skipping: [node1]

TASK [ambari_slave : include_tasks] ********************************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.049) 0:49:53.129 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_slave/tasks/install-agent-CentOS.yml for node1

TASK [ambari_slave : Install ambari-agent on CentOS 7] *************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.060) 0:49:53.190 *******
changed: [node1]

TASK [ambari_slave : Force Ambari Agent to use TLSv1.2] ************************
Friday 28 February 2020 21:50:08 +0500 (0:01:17.503) 0:51:10.693 *******
changed: [node1]

TASK [ambari_slave : include_tasks] ********************************************
Friday 28 February 2020 21:50:09 +0500 (0:00:00.743) 0:51:11.437 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_slave/tasks/hostname.yml for nod

TASK [ambari_slave : Create ambari-agent hostname script] **********************
Friday 28 February 2020 21:50:09 +0500 (0:00:00.059) 0:51:11.496 *******
changed: [node1]

TASK [ambari_slave : Configure ambari-server hostname in ambari-agent configuration] ***
Friday 28 February 2020 21:50:10 +0500 (0:00:01.149) 0:51:12.646 *******
changed: [node1] => (item={u'insertafter': u'\\[server\\]', u'regexp': u'^.*hostname=.*$', u'line': u'hostname=de1'})
changed: [node1] => (item={u'insertafter': u'\\[agent\\]', u'regexp': u'^hostname_script=.*$', u'line': u'hostne_script=/var/lib/ambari-agent/hostname.sh'})

TASK [ambari_slave : Ensure ambari-agent is running] ***************************
Friday 28 February 2020 21:50:11 +0500 (0:00:01.290) 0:51:13.936 *******
changed: [node1]

TASK [ambari_slave : Wait for agent to register] *******************************
Friday 28 February 2020 21:50:14 +0500 (0:00:03.090) 0:51:17.027 *******
changed: [node1]

PLAY [ec2] *********************************************************************
skipping: no hosts matched

PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [ambari_slave] ************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:50:25 +0500 (0:00:10.732) 0:51:27.759 *******
ok: [node1]

TASK [metron-packages : Validate support for distribution] *********************
Friday 28 February 2020 21:50:26 +0500 (0:00:00.966) 0:51:28.726 *******
skipping: [node1]

TASK [metron-packages : Clean out local repo] **********************************
Friday 28 February 2020 21:50:26 +0500 (0:00:00.045) 0:51:28.772 *******
ok: [node1]

TASK [metron-packages : Create local repo directory] ***************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.666) 0:51:29.438 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.709) 0:51:30.148 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.090) 0:51:30.239 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.046) 0:51:30.285 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/create_repo_centoyml for node1

TASK [metron-packages : Install tooling on CentOS] *****************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.064) 0:51:30.349 *******
changed: [node1]

TASK [metron-packages : Create local repo with new packages on CentOS] *********
Friday 28 February 2020 21:50:34 +0500 (0:00:06.684) 0:51:37.034 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:35 +0500 (0:00:00.839) 0:51:37.873 *******
skipping: [node1]

PLAY [metron] ******************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:50:35 +0500 (0:00:00.057) 0:51:37.931 *******
ok: [node1]

TASK [metron-packages : Validate support for distribution] *********************
Friday 28 February 2020 21:50:36 +0500 (0:00:00.955) 0:51:38.886 *******
skipping: [node1]

TASK [metron-packages : Clean out local repo] **********************************
Friday 28 February 2020 21:50:36 +0500 (0:00:00.049) 0:51:38.936 *******
changed: [node1]

TASK [metron-packages : Create local repo directory] ***************************
Friday 28 February 2020 21:50:37 +0500 (0:00:00.739) 0:51:39.675 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:37 +0500 (0:00:00.657) 0:51:40.333 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/copy_packages_cens.yml for node1

TASK [metron-packages : Copy Metron RPMs] **************************************
Friday 28 February 2020 21:50:38 +0500 (0:00:00.098) 0:51:40.431 *******
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-indexing-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-data-management-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-hbase-server-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-config-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-metron-management-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsers-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-solr-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-elasticsearch-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-maas-service-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-enrichment-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-stellar-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-alerts-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-performance-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-repl-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsers-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsing-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-pcap-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-rest-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-enrichment-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-solr-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-spark-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-indexing-common-0.7.2-202002281608.noarch.rpm)

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:32 +0500 (0:00:54.122) 0:52:34.554 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:32 +0500 (0:00:00.047) 0:52:34.601 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/create_repo_centoyml for node1

TASK [metron-packages : Install tooling on CentOS] *****************************
Friday 28 February 2020 21:51:32 +0500 (0:00:00.065) 0:52:34.667 *******
ok: [node1]

TASK [metron-packages : Create local repo with new packages on CentOS] *********
Friday 28 February 2020 21:51:33 +0500 (0:00:00.819) 0:52:35.487 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:35 +0500 (0:00:02.567) 0:52:38.054 *******
skipping: [node1]

PLAY [ambari_master] ***********************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:51:35 +0500 (0:00:00.058) 0:52:38.113 *******
ok: [node1]

TASK [epel : Install EPEL repository] ******************************************
Friday 28 February 2020 21:51:36 +0500 (0:00:01.035) 0:52:39.149 *******
ok: [node1]

TASK [python-pip : Install Python's pip on Centos] *****************************
Friday 28 February 2020 21:51:37 +0500 (0:00:00.756) 0:52:39.905 *******
changed: [node1]

TASK [python-pip : Install Python's pip on Ubuntu] *****************************
Friday 28 February 2020 21:51:46 +0500 (0:00:08.579) 0:52:48.484 *******
skipping: [node1]

TASK [httplib2 : Install python httplib2 dependency] ***************************
Friday 28 February 2020 21:51:46 +0500 (0:00:00.062) 0:52:48.547 *******
changed: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:51:52 +0500 (0:00:05.977) 0:52:54.525 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.073) 0:52:54.599 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.654) 0:52:55.253 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.067) 0:52:55.321 *******
ok: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.819) 0:52:56.140 *******
skipping: [node1]

TASK [ambari_config : include_vars] ********************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.049) 0:52:56.190 *******
ok: [node1]

TASK [ambari_config : include_tasks] *******************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.127) 0:52:56.317 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_config/tasks/dependencies-CentOS.yml for node1

TASK [ambari_config : Install Python dependencies] *****************************
Friday 28 February 2020 21:51:54 +0500 (0:00:00.061) 0:52:56.379 *******
changed: [node1] => (item=[u'python-urllib3', u'python-requests'])

TASK [ambari_config : Wait for Ambari to start; http://node1:8080] *************
Friday 28 February 2020 21:52:02 +0500 (0:00:08.183) 0:53:04.563 *******
ok: [node1]

TASK [ambari_config : Deploy cluster with Ambari; http://node1:8080] ***********
Friday 28 February 2020 21:52:02 +0500 (0:00:00.794) 0:53:05.357 *******
[WARNING]: Module did not set no_log for password

fatal: [node1]: FAILED! => {"changed": false, "msg": "Request failed with status FAILED"}
to retry, use: --limit @/home/abidzaidi/Source/metron/metron-deployment/development/centos7/ansible/plaook.retry

PLAY RECAP *********************************************************************
node1 : ok=99 changed=53 unreachable=0 failed=1

Friday 28 February 2020 22:02:20 +0500 (0:10:17.743) 1:03:23.100 *******
===============================================================================
metron-builder : Build Metron RPM Packages --------------------------- 841.29s
ambari_master : Install Ambari Server on CentOS ----------------------- 699.73s
ambari_config : Deploy cluster with Ambari; http://node1:8080 --------- 617.74s
metron-builder : Build Metron ----------------------------------------- 496.19s
ambari_master : Setup Ambari Server ----------------------------------- 348.15s
solr : Download Solr -------------------------------------------------- 281.83s
java_jdk : Install openjdk -------------------------------------------- 109.26s
ambari_slave : Install ambari-agent on CentOS 7 ------------------------ 77.50s
ambari_common : Update package cache on CentOS ------------------------- 66.31s
metron-packages : Copy Metron RPMs ------------------------------------- 54.12s
ntp : Install ntp ------------------------------------------------------ 28.42s
ambari_common : Install Node.js repo on CentOS 7 ----------------------- 24.47s
ambari_master : start ambari server ------------------------------------ 19.38s
enable-swap : Allocate 8092 mb for swap space -------------------------- 18.72s
ambari_slave : Wait for agent to register ------------------------------ 10.73s
python-pip : Install Python's pip on Centos ----------------------------- 8.58s
ambari_config : Install Python dependencies ----------------------------- 8.18s
enable-swap : Setup swap space ------------------------------------------ 7.21s
metron-builder : Clean Metron ------------------------------------------- 6.88s
metron-packages : Install tooling on CentOS ----------------------------- 6.68s
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again

 

After trying provision with following command,

env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision

 

Spoiler

PLAY [ambari_master] ***********************************************************

.

.

.

.

The full traceback is:
File "/tmp/ansible_VXhm7E/ansible_module_yum.py", line 565, in what_provides
pkgs = my.returnPackagesByDep(req_spec) + my.returnInstalledPackagesByDep(req_spec)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 4298, in returnPackagesByDep
return self.pkgSack.searchProvides(depstring)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1075, in <lambda>
pkgSack = property(fget=lambda self: self._getSacks(),
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks
self.repos.populateSack(which=repos)
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 347, in populateSack
self.doSetup()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
self.retrieveAllMD()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
dl = repo._async and repo._commonLoadRepoXML(repo)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1482, in _commonLoadRepoXML
result = self._getFileRepoXML(local, text)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1259, in _getFileRepoXML
size=102400) # setting max size as 100K
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1042, in _getFile
raise e

fatal: [node1]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"allow_downgrade": false,
"bugfix": false,
"conf_file": null,
"disable_gpg_check": false,
"disable_plugin": [],
"disablerepo": null,
"enable_plugin": [],
"enablerepo": null,
"exclude": null,
"install_repoquery": true,
"installroot": "/",
"list": null,
"name": [
"python-pip"
],
"security": false,
"skip_broken": false,
"state": "installed",
"update_cache": false,
"update_only": false,
"validate_certs": true
}
},

"msg": "Failure talking to yum: failure: repodata/repomd.xml from HDP-3.1-repo-1: [Errno 256] No more mirrors to try.\nhttps://archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0/repodata/repomd.xml: [Errno 14] HTTPS Error 401 - Unauthorized"
}
to retry, use: --limit @/home/abidzaidi/Source/metron/metron-deployment/development/centos7/ansible/playbook.retry

I need to setup a Dev Environment where I could add a few more features in Apache Metron's REST API with UI pages on the front-end. I seem to be far away from my targets as I am unable to start up the vagrant machine, It would be good if a Metron Developer could guide me with what I'm trying to achieve. 

Regards, Abid Zaidi. 

 

1 REPLY 1
Highlighted

Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Single Node on HDP 3.1 Branch)

New Contributor

Here is the output of my platform.sh

Spoiler

abidzaidi@abidzaidi:~/Source/metron$ metron-deployment/scripts/platform-info.sh
Metron 0.7.2
--
* feature/METRON-2088-support-hdp-3.1
--
commit 889f113f51e9d57b6189ee32f6eec6461a19db9c (HEAD -> feature/METRON-2088-support-hdp-3.1, origin/feature/METRON-2088-support-hdp-3.1)
Author: nickwallen <nickallen@apache.org>
Date: Mon Nov 18 09:54:58 2019 -0500

METRON-2314 HDFSIndexingIntegrationTests Fails with SLF4J/Log4j Delegation Loop (nickwallen) closes apache/metron#1559
--
metron-deployment/development/centos7/Vagrantfile | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
ansible 2.6.5
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/abidzaidi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.17 (default, Nov 7 2019, 10:07:09) [GCC 9.2.1 20191008]
--
Vagrant 2.2.7
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.17
--
Apache Maven 3.6.1
Maven home: /usr/share/maven
Java version: 1.8.0_242, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.0-40-generic", arch: "amd64", family: "unix"
--
Docker version 19.03.6, build 369ce74a3c
--
node
v13.9.0
--
npm
6.13.7
--
g++ (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

--
Compiler is C++11 compliant
--
Linux abidzaidi 5.3.0-40-generic #32-Ubuntu SMP Fri Jan 31 20:24:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 31785.1 MB
Processor Model: Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
Processor Speed: 1907.347 MHz
Processor Speed: 1269.641 MHz
Processor Speed: 1431.551 MHz
Processor Speed: 1200.024 MHz
Processor Speed: 2821.605 MHz
Processor Speed: 3106.129 MHz
Processor Speed: 1215.431 MHz
Processor Speed: 1212.018 MHz
Processor Speed: 2045.421 MHz
Processor Speed: 1200.047 MHz
Processor Speed: 1275.653 MHz
Processor Speed: 1884.051 MHz
Total Physical Processors: 12
Total cores: 72
Disk information:
/dev/sda1 229G 40G 178G 19% /
/dev/loop1 45M 45M 0 100% /snap/gtk-common-themes/1353
/dev/loop2 45M 45M 0 100% /snap/gtk-common-themes/1440
/dev/loop3 164M 164M 0 100% /snap/spotify/41
/dev/loop5 15M 15M 0 100% /snap/gnome-characters/317
/dev/loop4 153M 153M 0 100% /snap/chromium/1040
/dev/loop0 150M 150M 0 100% /snap/gnome-3-28-1804/71
/dev/loop6 92M 92M 0 100% /snap/core/8689
/dev/loop7 55M 55M 0 100% /snap/core18/1223
/dev/loop8 1.0M 1.0M 0 100% /snap/gnome-logs/81
/dev/loop9 135M 135M 0 100% /snap/code/25
/dev/loop10 161M 161M 0 100% /snap/gnome-3-28-1804/116
/dev/loop11 4.3M 4.3M 0 100% /snap/gnome-calculator/501
/dev/loop13 4.3M 4.3M 0 100% /snap/gnome-calculator/544
/dev/loop14 15M 15M 0 100% /snap/gnome-characters/399
/dev/loop15 90M 90M 0 100% /snap/core/7917
/dev/loop12 728M 728M 0 100% /snap/intellij-idea-ultimate/204
/dev/loop16 55M 55M 0 100% /snap/core18/1668
This CPU appears to support virtualization

Don't have an account?
Coming from Hortonworks? Activate your account here