Support Questions
Find answers, ask questions, and share your expertise

Failing to Setup Apache Metron Dev Machine using Vagrant (Single Node on HDP 3.1 Branch)

New Contributor

I am setting up centos7, I already tried centos6 from other branches and got no luck in that, so I took a branch of metron hdp3.1 and followed the ReadMe, before starting I ran platform.sh and everything looked good..

On Vagrant Up it just fails on Metron_config with message

[node1]: FAILED! => {"changed": false, "msg": "Request failed with status FAILED"}

At this point I could open ambari but all services were down and each of them said the same thing, Failed to install service, Below are my logs

Spoiler

abidzaidi@abidzaidi:~/Source/metron/metron-deployment/development/centos7$ vagrant up
Running with ansible-skip-tags: ["sensors"]
Bringing machine 'node1' up with 'virtualbox' provider...
==> node1: Importing base box 'centos/7'...
==> node1: Matching MAC address for NAT networking...
==> node1: Checking if box 'centos/7' version '1905.1' is up to date...
==> node1: Setting the name of the VM: centos7_node1_1582905500842_94659
==> node1: Clearing any previously set network interfaces...
==> node1: Preparing network interfaces based on configuration...
node1: Adapter 1: nat
node1: Adapter 2: hostonly
==> node1: Forwarding ports...
node1: 22 (guest) => 2222 (host) (adapter 1)
==> node1: Running 'pre-boot' VM customizations...
==> node1: Booting VM...
==> node1: Waiting for machine to boot. This may take a few minutes...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
node1:
node1: Vagrant insecure key detected. Vagrant will automatically replace
node1: this with a newly generated keypair for better security.
node1:
node1: Inserting generated public key within guest...
node1: Removing insecure key from the guest if it's present...
node1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> node1: Machine booted and ready!
==> node1: Checking for guest additions in VM...
node1: No guest additions were detected on the base box for this VM! Guest
node1: additions are required for forwarded ports, shared folders, host only
node1: networking, and more. If SSH fails on this machine, please install
node1: the guest additions and repackage the box to continue.
node1:
node1: This is not an error message; everything may continue to work properly,
node1: in which case you may ignore this message.
==> node1: Setting hostname...
==> node1: Configuring and enabling network interfaces...
==> node1: Rsyncing folder: /home/abidzaidi/Source/metron/metron-deployment/development/centos7/ => /vagrant
==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...
==> node1: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.6.5).

Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode

node1: Running ansible-playbook...

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 20:58:57 +0500 (0:00:00.052) 0:00:00.052 *******
ok: [node1]

TASK [libselinux-python : Install libselinux-python] ***************************
Friday 28 February 2020 20:58:59 +0500 (0:00:02.009) 0:00:02.061 *******
ok: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:00 +0500 (0:00:00.890) 0:00:02.952 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/check-swap.yml for no1

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:00 +0500 (0:00:00.049) 0:00:03.002 *******
changed: [node1]

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.685) 0:00:03.687 *******
changed: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.625) 0:00:04.313 *******
skipping: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:01 +0500 (0:00:00.047) 0:00:04.360 *******
ok: [node1] => {
"msg": "Swap is NOT enabled"
}

TASK [enable-swap : Checking for /swappy] **************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.096) 0:00:04.457 *******
ok: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.656) 0:00:05.113 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/setup-swap.yml for no1

TASK [enable-swap : Disable swap space] ****************************************
Friday 28 February 2020 20:59:02 +0500 (0:00:00.071) 0:00:05.185 *******
changed: [node1]

TASK [enable-swap : Allocate 8092 mb for swap space] ***************************
Friday 28 February 2020 20:59:03 +0500 (0:00:00.781) 0:00:05.967 *******
changed: [node1]

TASK [enable-swap : Permissioning /swappy] *************************************
Friday 28 February 2020 20:59:22 +0500 (0:00:18.719) 0:00:24.687 *******
changed: [node1]

TASK [enable-swap : Setup swap space] ******************************************
Friday 28 February 2020 20:59:23 +0500 (0:00:00.764) 0:00:25.451 *******
changed: [node1]

TASK [enable-swap : Enable swap space] *****************************************
Friday 28 February 2020 20:59:30 +0500 (0:00:07.207) 0:00:32.659 *******
changed: [node1]

TASK [enable-swap : Enable swap on boot] ***************************************
Friday 28 February 2020 20:59:31 +0500 (0:00:01.208) 0:00:33.868 *******
changed: [node1]

TASK [enable-swap : include_tasks] *********************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.730) 0:00:34.599 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/enable-swap/tasks/check-swap.yml for no1

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.051) 0:00:34.650 *******
changed: [node1]

TASK [enable-swap : shell] *****************************************************
Friday 28 February 2020 20:59:32 +0500 (0:00:00.623) 0:00:35.274 *******
changed: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.620) 0:00:35.894 *******
ok: [node1] => {
"msg": "Swap is enabled with 8091 mb"
}

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.101) 0:00:35.996 *******
skipping: [node1]

TASK [enable-swap : debug] *****************************************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.045) 0:00:36.042 *******
skipping: [node1]

TASK [enable-remote-ssh : Comment password auth no] ****************************
Friday 28 February 2020 20:59:33 +0500 (0:00:00.042) 0:00:36.084 *******
changed: [node1]

TASK [enable-remote-ssh : Uncomment password auth yes] *************************
Friday 28 February 2020 20:59:34 +0500 (0:00:00.611) 0:00:36.696 *******
changed: [node1]

TASK [enable-remote-ssh : Restart service sshd, in all cases] ******************
Friday 28 February 2020 20:59:34 +0500 (0:00:00.628) 0:00:37.324 *******
changed: [node1]

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 20:59:35 +0500 (0:00:00.788) 0:00:38.112 *******
ok: [node1]

TASK [metron-builder : Check distribution] *************************************
Friday 28 February 2020 20:59:36 +0500 (0:00:00.861) 0:00:38.974 *******
skipping: [node1]

TASK [metron-builder : Clean Metron] *******************************************
Friday 28 February 2020 20:59:36 +0500 (0:00:00.053) 0:00:39.027 *******
changed: [node1 -> localhost] => (item=mvn clean -P HDP-3.1)
changed: [node1 -> localhost] => (item=mvn clean -P mpack)
changed: [node1 -> localhost] => (item=mvn clean -P build-rpms)
changed: [node1 -> localhost] => (item=mvn clean -P build-debs)

TASK [metron-builder : Build Metron] *******************************************
Friday 28 February 2020 20:59:43 +0500 (0:00:06.876) 0:00:45.904 *******
changed: [node1 -> localhost] => (item=mvn package -DskipTests -T 2C -P HDP-3.1,mpack)

TASK [metron-builder : include_tasks] ******************************************
Friday 28 February 2020 21:07:59 +0500 (0:08:16.190) 0:09:02.095 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-builder/tasks/build-rpms.yml forode1

TASK [metron-builder : Build Metron RPM Packages] ******************************
Friday 28 February 2020 21:07:59 +0500 (0:00:00.067) 0:09:02.162 *******
^[[C^[[D
changed: [node1 -> localhost] => (item=mvn package -DskipTests -Pbuild-rpms -T 2C)

TASK [metron-builder : include_tasks] ******************************************
Friday 28 February 2020 21:22:01 +0500 (0:14:01.289) 0:23:03.451 *******
skipping: [node1]
[WARNING]: Could not match supplied host pattern, ignoring: ec2


PLAY [ec2] *********************************************************************
skipping: no hosts matched
[WARNING]: Could not match supplied host pattern, ignoring: packer


PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [search] ******************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:22:01 +0500 (0:00:00.061) 0:23:03.513 *******
ok: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:22:02 +0500 (0:00:01.858) 0:23:05.372 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.096) 0:23:05.468 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.820) 0:23:06.289 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:22:03 +0500 (0:00:00.069) 0:23:06.358 *******
changed: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:23:53 +0500 (0:01:49.258) 0:24:55.617 *******
skipping: [node1]

TASK [solr : Create Solr user] *************************************************
Friday 28 February 2020 21:23:53 +0500 (0:00:00.051) 0:24:55.668 *******
changed: [node1]

TASK [solr : Download Solr] ****************************************************
Friday 28 February 2020 21:23:54 +0500 (0:00:00.833) 0:24:56.502 *******
changed: [node1]

TASK [solr : Extract Solr tarball] *********************************************
Friday 28 February 2020 21:28:35 +0500 (0:04:41.828) 0:29:38.330 *******
changed: [node1]

PLAY [ec2] *********************************************************************
skipping: no hosts matched

PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [ambari_*] ****************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:28:42 +0500 (0:00:06.200) 0:29:44.530 *******
ok: [node1]

TASK [libselinux-python : Install libselinux-python] ***************************
Friday 28 February 2020 21:28:43 +0500 (0:00:00.981) 0:29:45.512 *******
ok: [node1]

TASK [epel : Install EPEL repository] ******************************************
Friday 28 February 2020 21:28:44 +0500 (0:00:00.942) 0:29:46.454 *******
changed: [node1]

TASK [ntp : set_fact] **********************************************************
Friday 28 February 2020 21:28:46 +0500 (0:00:02.860) 0:29:49.315 *******
ok: [node1]

TASK [ntp : set_fact] **********************************************************
Friday 28 February 2020 21:28:47 +0500 (0:00:00.129) 0:29:49.444 *******
skipping: [node1]

TASK [ntp : Install ntp] *******************************************************
Friday 28 February 2020 21:28:47 +0500 (0:00:00.069) 0:29:49.514 *******
changed: [node1]

TASK [ntp : Ensure ntp is running and enabled] *********************************
Friday 28 February 2020 21:29:15 +0500 (0:00:28.419) 0:30:17.933 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:16 +0500 (0:00:00.876) 0:30:18.809 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/firewall-CentOS-7.y for node1

TASK [ambari_common : Stop firewalld on CentOS 7] ******************************
Friday 28 February 2020 21:29:16 +0500 (0:00:00.061) 0:30:18.871 *******
ok: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:17 +0500 (0:00:00.656) 0:30:19.527 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/hostname.yml for no1

TASK [ambari_common : Remove ipv4 'localhost.localdomain' from /etc/hosts] *****
Friday 28 February 2020 21:29:17 +0500 (0:00:00.058) 0:30:19.586 *******
changed: [node1]

TASK [ambari_common : Remove ipv6 'localhost.localdomain' from /etc/hosts] *****
Friday 28 February 2020 21:29:17 +0500 (0:00:00.621) 0:30:20.207 *******
changed: [node1]

TASK [ambari_common : Add localhost to /etc/hosts] *****************************
Friday 28 February 2020 21:29:18 +0500 (0:00:00.636) 0:30:20.844 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:29:19 +0500 (0:00:00.664) 0:30:21.509 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/ambari-repo-CentOS-yml for node1

TASK [ambari_common : Setup Ambari repo on CentOS] *****************************
Friday 28 February 2020 21:29:19 +0500 (0:00:00.061) 0:30:21.570 *******
changed: [node1]

TASK [ambari_common : Update package cache on CentOS] **************************
Friday 28 February 2020 21:29:20 +0500 (0:00:01.492) 0:30:23.062 *******
ok: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:30:26 +0500 (0:01:06.309) 0:31:29.372 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/nodejs-CentOS-7.ymlor node1

TASK [ambari_common : Install Node.js repo on CentOS 7] ************************
Friday 28 February 2020 21:30:27 +0500 (0:00:00.059) 0:31:29.432 *******
changed: [node1]

TASK [ambari_common : include_tasks] *******************************************
Friday 28 February 2020 21:30:51 +0500 (0:00:24.472) 0:31:53.905 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_common/tasks/logrotate.yml for ne1

TASK [ambari_common : Create Logrotate Script for Hadoop Services] *************
Friday 28 February 2020 21:30:51 +0500 (0:00:00.052) 0:31:53.958 *******
changed: [node1]

PLAY [ambari_master] ***********************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:30:52 +0500 (0:00:01.196) 0:31:55.155 *******
ok: [node1]

TASK [ambari_master : Install Ambari Server on CentOS] *************************
Friday 28 February 2020 21:30:53 +0500 (0:00:00.940) 0:31:56.096 *******
changed: [node1]

TASK [ambari_master : Install Ambari Server on Ubuntu] *************************
Friday 28 February 2020 21:42:33 +0500 (0:11:39.728) 0:43:35.824 *******
skipping: [node1]

TASK [ambari_master : Set Ambari Server Max Memory] ****************************
Friday 28 February 2020 21:42:33 +0500 (0:00:00.043) 0:43:35.867 *******
ok: [node1]

TASK [ambari_master : Allow 150 seconds of startup time for ambari server] *****
Friday 28 February 2020 21:42:34 +0500 (0:00:00.751) 0:43:36.619 *******
changed: [node1]

TASK [ambari_master : Setup Ambari Server] *************************************
Friday 28 February 2020 21:42:35 +0500 (0:00:01.260) 0:43:37.879 *******
changed: [node1]

TASK [ambari_master : Copy Metron MPack to Ambari Host] ************************
Friday 28 February 2020 21:48:23 +0500 (0:05:48.149) 0:49:26.028 *******
changed: [node1]

TASK [ambari_master : Install Metron MPack on Ambari Host] *********************
Friday 28 February 2020 21:48:25 +0500 (0:00:01.534) 0:49:27.562 *******
changed: [node1]

TASK [ambari_master : Copy Elasticsearch MPack to Ambari Host] *****************
Friday 28 February 2020 21:48:26 +0500 (0:00:01.142) 0:49:28.704 *******
changed: [node1]

TASK [ambari_master : Install Elasticsearch MPack on Ambari Host] **************
Friday 28 February 2020 21:48:27 +0500 (0:00:01.526) 0:49:30.231 *******
changed: [node1]

TASK [ambari_master : start ambari server] *************************************
Friday 28 February 2020 21:48:28 +0500 (0:00:01.106) 0:49:31.338 *******
changed: [node1]

PLAY [ambari_slave] ************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:48:48 +0500 (0:00:19.375) 0:49:50.713 *******
ok: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.886) 0:49:51.600 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.077) 0:49:51.677 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.562) 0:49:52.240 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:48:49 +0500 (0:00:00.064) 0:49:52.304 *******
ok: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.775) 0:49:53.080 *******
skipping: [node1]

TASK [ambari_slave : include_tasks] ********************************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.049) 0:49:53.129 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_slave/tasks/install-agent-CentOS.yml for node1

TASK [ambari_slave : Install ambari-agent on CentOS 7] *************************
Friday 28 February 2020 21:48:50 +0500 (0:00:00.060) 0:49:53.190 *******
changed: [node1]

TASK [ambari_slave : Force Ambari Agent to use TLSv1.2] ************************
Friday 28 February 2020 21:50:08 +0500 (0:01:17.503) 0:51:10.693 *******
changed: [node1]

TASK [ambari_slave : include_tasks] ********************************************
Friday 28 February 2020 21:50:09 +0500 (0:00:00.743) 0:51:11.437 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_slave/tasks/hostname.yml for nod

TASK [ambari_slave : Create ambari-agent hostname script] **********************
Friday 28 February 2020 21:50:09 +0500 (0:00:00.059) 0:51:11.496 *******
changed: [node1]

TASK [ambari_slave : Configure ambari-server hostname in ambari-agent configuration] ***
Friday 28 February 2020 21:50:10 +0500 (0:00:01.149) 0:51:12.646 *******
changed: [node1] => (item={u'insertafter': u'\\[server\\]', u'regexp': u'^.*hostname=.*$', u'line': u'hostname=de1'})
changed: [node1] => (item={u'insertafter': u'\\[agent\\]', u'regexp': u'^hostname_script=.*$', u'line': u'hostne_script=/var/lib/ambari-agent/hostname.sh'})

TASK [ambari_slave : Ensure ambari-agent is running] ***************************
Friday 28 February 2020 21:50:11 +0500 (0:00:01.290) 0:51:13.936 *******
changed: [node1]

TASK [ambari_slave : Wait for agent to register] *******************************
Friday 28 February 2020 21:50:14 +0500 (0:00:03.090) 0:51:17.027 *******
changed: [node1]

PLAY [ec2] *********************************************************************
skipping: no hosts matched

PLAY [packer] ******************************************************************
skipping: no hosts matched

PLAY [ambari_slave] ************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:50:25 +0500 (0:00:10.732) 0:51:27.759 *******
ok: [node1]

TASK [metron-packages : Validate support for distribution] *********************
Friday 28 February 2020 21:50:26 +0500 (0:00:00.966) 0:51:28.726 *******
skipping: [node1]

TASK [metron-packages : Clean out local repo] **********************************
Friday 28 February 2020 21:50:26 +0500 (0:00:00.045) 0:51:28.772 *******
ok: [node1]

TASK [metron-packages : Create local repo directory] ***************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.666) 0:51:29.438 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.709) 0:51:30.148 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.090) 0:51:30.239 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.046) 0:51:30.285 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/create_repo_centoyml for node1

TASK [metron-packages : Install tooling on CentOS] *****************************
Friday 28 February 2020 21:50:27 +0500 (0:00:00.064) 0:51:30.349 *******
changed: [node1]

TASK [metron-packages : Create local repo with new packages on CentOS] *********
Friday 28 February 2020 21:50:34 +0500 (0:00:06.684) 0:51:37.034 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:35 +0500 (0:00:00.839) 0:51:37.873 *******
skipping: [node1]

PLAY [metron] ******************************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:50:35 +0500 (0:00:00.057) 0:51:37.931 *******
ok: [node1]

TASK [metron-packages : Validate support for distribution] *********************
Friday 28 February 2020 21:50:36 +0500 (0:00:00.955) 0:51:38.886 *******
skipping: [node1]

TASK [metron-packages : Clean out local repo] **********************************
Friday 28 February 2020 21:50:36 +0500 (0:00:00.049) 0:51:38.936 *******
changed: [node1]

TASK [metron-packages : Create local repo directory] ***************************
Friday 28 February 2020 21:50:37 +0500 (0:00:00.739) 0:51:39.675 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:50:37 +0500 (0:00:00.657) 0:51:40.333 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/copy_packages_cens.yml for node1

TASK [metron-packages : Copy Metron RPMs] **************************************
Friday 28 February 2020 21:50:38 +0500 (0:00:00.098) 0:51:40.431 *******
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-indexing-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-data-management-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-hbase-server-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-config-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-metron-management-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsers-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-solr-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-elasticsearch-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-maas-service-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-enrichment-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-stellar-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-alerts-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-performance-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-repl-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsers-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-parsing-storm-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-pcap-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-rest-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-enrichment-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-solr-common-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-profiler-spark-0.7.2-202002281608.noarch.rpm)
changed: [node1] => (item=/home/abidzaidi/Source/metron/metron-deployment/packaging/docker/rpm-docker/target/RP/noarch/metron-indexing-common-0.7.2-202002281608.noarch.rpm)

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:32 +0500 (0:00:54.122) 0:52:34.554 *******
skipping: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:32 +0500 (0:00:00.047) 0:52:34.601 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/metron-packages/tasks/create_repo_centoyml for node1

TASK [metron-packages : Install tooling on CentOS] *****************************
Friday 28 February 2020 21:51:32 +0500 (0:00:00.065) 0:52:34.667 *******
ok: [node1]

TASK [metron-packages : Create local repo with new packages on CentOS] *********
Friday 28 February 2020 21:51:33 +0500 (0:00:00.819) 0:52:35.487 *******
changed: [node1]

TASK [metron-packages : include_tasks] *****************************************
Friday 28 February 2020 21:51:35 +0500 (0:00:02.567) 0:52:38.054 *******
skipping: [node1]

PLAY [ambari_master] ***********************************************************

TASK [Gathering Facts] *********************************************************
Friday 28 February 2020 21:51:35 +0500 (0:00:00.058) 0:52:38.113 *******
ok: [node1]

TASK [epel : Install EPEL repository] ******************************************
Friday 28 February 2020 21:51:36 +0500 (0:00:01.035) 0:52:39.149 *******
ok: [node1]

TASK [python-pip : Install Python's pip on Centos] *****************************
Friday 28 February 2020 21:51:37 +0500 (0:00:00.756) 0:52:39.905 *******
changed: [node1]

TASK [python-pip : Install Python's pip on Ubuntu] *****************************
Friday 28 February 2020 21:51:46 +0500 (0:00:08.579) 0:52:48.484 *******
skipping: [node1]

TASK [httplib2 : Install python httplib2 dependency] ***************************
Friday 28 February 2020 21:51:46 +0500 (0:00:00.062) 0:52:48.547 *******
changed: [node1]

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:51:52 +0500 (0:00:05.977) 0:52:54.525 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/java_jdk/tasks/install_jdk_centos.yml f node1

TASK [java_jdk : Check for java at "/usr/jdk64/jdk1.8.0_77"] *******************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.073) 0:52:54.599 *******
ok: [node1]

TASK [java_jdk : Alternatives link for java] ***********************************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.654) 0:52:55.253 *******
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/java', u'link': u'/usr/bin/java', u'name': u'va'})
skipping: [node1] => (item={u'path': u'/usr/jdk64/jdk1.8.0_77/bin/jar', u'link': u'/usr/bin/jar', u'name': u'ja})

TASK [java_jdk : Install openjdk] **********************************************
Friday 28 February 2020 21:51:52 +0500 (0:00:00.067) 0:52:55.321 *******
ok: [node1] => (item=[u'java-1.8.0-openjdk', u'java-1.8.0-openjdk-devel'])

TASK [java_jdk : include_tasks] ************************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.819) 0:52:56.140 *******
skipping: [node1]

TASK [ambari_config : include_vars] ********************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.049) 0:52:56.190 *******
ok: [node1]

TASK [ambari_config : include_tasks] *******************************************
Friday 28 February 2020 21:51:53 +0500 (0:00:00.127) 0:52:56.317 *******
included: /home/abidzaidi/Source/metron/metron-deployment/ansible/roles/ambari_config/tasks/dependencies-CentOS.yml for node1

TASK [ambari_config : Install Python dependencies] *****************************
Friday 28 February 2020 21:51:54 +0500 (0:00:00.061) 0:52:56.379 *******
changed: [node1] => (item=[u'python-urllib3', u'python-requests'])

TASK [ambari_config : Wait for Ambari to start; http://node1:8080] *************
Friday 28 February 2020 21:52:02 +0500 (0:00:08.183) 0:53:04.563 *******
ok: [node1]

TASK [ambari_config : Deploy cluster with Ambari; http://node1:8080] ***********
Friday 28 February 2020 21:52:02 +0500 (0:00:00.794) 0:53:05.357 *******
[WARNING]: Module did not set no_log for password

fatal: [node1]: FAILED! => {"changed": false, "msg": "Request failed with status FAILED"}
to retry, use: --limit @/home/abidzaidi/Source/metron/metron-deployment/development/centos7/ansible/plaook.retry

PLAY RECAP *********************************************************************
node1 : ok=99 changed=53 unreachable=0 failed=1

Friday 28 February 2020 22:02:20 +0500 (0:10:17.743) 1:03:23.100 *******
===============================================================================
metron-builder : Build Metron RPM Packages --------------------------- 841.29s
ambari_master : Install Ambari Server on CentOS ----------------------- 699.73s
ambari_config : Deploy cluster with Ambari; http://node1:8080 --------- 617.74s
metron-builder : Build Metron ----------------------------------------- 496.19s
ambari_master : Setup Ambari Server ----------------------------------- 348.15s
solr : Download Solr -------------------------------------------------- 281.83s
java_jdk : Install openjdk -------------------------------------------- 109.26s
ambari_slave : Install ambari-agent on CentOS 7 ------------------------ 77.50s
ambari_common : Update package cache on CentOS ------------------------- 66.31s
metron-packages : Copy Metron RPMs ------------------------------------- 54.12s
ntp : Install ntp ------------------------------------------------------ 28.42s
ambari_common : Install Node.js repo on CentOS 7 ----------------------- 24.47s
ambari_master : start ambari server ------------------------------------ 19.38s
enable-swap : Allocate 8092 mb for swap space -------------------------- 18.72s
ambari_slave : Wait for agent to register ------------------------------ 10.73s
python-pip : Install Python's pip on Centos ----------------------------- 8.58s
ambari_config : Install Python dependencies ----------------------------- 8.18s
enable-swap : Setup swap space ------------------------------------------ 7.21s
metron-builder : Clean Metron ------------------------------------------- 6.88s
metron-packages : Install tooling on CentOS ----------------------------- 6.68s
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again

 

After trying provision with following command,

env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision

 

Spoiler

PLAY [ambari_master] ***********************************************************

.

.

.

.

The full traceback is:
File "/tmp/ansible_VXhm7E/ansible_module_yum.py", line 565, in what_provides
pkgs = my.returnPackagesByDep(req_spec) + my.returnInstalledPackagesByDep(req_spec)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 4298, in returnPackagesByDep
return self.pkgSack.searchProvides(depstring)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1075, in <lambda>
pkgSack = property(fget=lambda self: self._getSacks(),
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks
self.repos.populateSack(which=repos)
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 347, in populateSack
self.doSetup()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
self.retrieveAllMD()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
dl = repo._async and repo._commonLoadRepoXML(repo)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1482, in _commonLoadRepoXML
result = self._getFileRepoXML(local, text)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1259, in _getFileRepoXML
size=102400) # setting max size as 100K
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1042, in _getFile
raise e

fatal: [node1]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"allow_downgrade": false,
"bugfix": false,
"conf_file": null,
"disable_gpg_check": false,
"disable_plugin": [],
"disablerepo": null,
"enable_plugin": [],
"enablerepo": null,
"exclude": null,
"install_repoquery": true,
"installroot": "/",
"list": null,
"name": [
"python-pip"
],
"security": false,
"skip_broken": false,
"state": "installed",
"update_cache": false,
"update_only": false,
"validate_certs": true
}
},

"msg": "Failure talking to yum: failure: repodata/repomd.xml from HDP-3.1-repo-1: [Errno 256] No more mirrors to try.\nhttps://archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0/repodata/repomd.xml: [Errno 14] HTTPS Error 401 - Unauthorized"
}
to retry, use: --limit @/home/abidzaidi/Source/metron/metron-deployment/development/centos7/ansible/playbook.retry

I need to setup a Dev Environment where I could add a few more features in Apache Metron's REST API with UI pages on the front-end. I seem to be far away from my targets as I am unable to start up the vagrant machine, It would be good if a Metron Developer could guide me with what I'm trying to achieve. 

Regards, Abid Zaidi. 

 

1 REPLY 1

New Contributor

Here is the output of my platform.sh

Spoiler

abidzaidi@abidzaidi:~/Source/metron$ metron-deployment/scripts/platform-info.sh
Metron 0.7.2
--
* feature/METRON-2088-support-hdp-3.1
--
commit 889f113f51e9d57b6189ee32f6eec6461a19db9c (HEAD -> feature/METRON-2088-support-hdp-3.1, origin/feature/METRON-2088-support-hdp-3.1)
Author: nickwallen <nickallen@apache.org>
Date: Mon Nov 18 09:54:58 2019 -0500

METRON-2314 HDFSIndexingIntegrationTests Fails with SLF4J/Log4j Delegation Loop (nickwallen) closes apache/metron#1559
--
metron-deployment/development/centos7/Vagrantfile | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
ansible 2.6.5
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/abidzaidi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.17 (default, Nov 7 2019, 10:07:09) [GCC 9.2.1 20191008]
--
Vagrant 2.2.7
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.17
--
Apache Maven 3.6.1
Maven home: /usr/share/maven
Java version: 1.8.0_242, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.0-40-generic", arch: "amd64", family: "unix"
--
Docker version 19.03.6, build 369ce74a3c
--
node
v13.9.0
--
npm
6.13.7
--
g++ (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

--
Compiler is C++11 compliant
--
Linux abidzaidi 5.3.0-40-generic #32-Ubuntu SMP Fri Jan 31 20:24:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
--
Total System Memory = 31785.1 MB
Processor Model: Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
Processor Speed: 1907.347 MHz
Processor Speed: 1269.641 MHz
Processor Speed: 1431.551 MHz
Processor Speed: 1200.024 MHz
Processor Speed: 2821.605 MHz
Processor Speed: 3106.129 MHz
Processor Speed: 1215.431 MHz
Processor Speed: 1212.018 MHz
Processor Speed: 2045.421 MHz
Processor Speed: 1200.047 MHz
Processor Speed: 1275.653 MHz
Processor Speed: 1884.051 MHz
Total Physical Processors: 12
Total cores: 72
Disk information:
/dev/sda1 229G 40G 178G 19% /
/dev/loop1 45M 45M 0 100% /snap/gtk-common-themes/1353
/dev/loop2 45M 45M 0 100% /snap/gtk-common-themes/1440
/dev/loop3 164M 164M 0 100% /snap/spotify/41
/dev/loop5 15M 15M 0 100% /snap/gnome-characters/317
/dev/loop4 153M 153M 0 100% /snap/chromium/1040
/dev/loop0 150M 150M 0 100% /snap/gnome-3-28-1804/71
/dev/loop6 92M 92M 0 100% /snap/core/8689
/dev/loop7 55M 55M 0 100% /snap/core18/1223
/dev/loop8 1.0M 1.0M 0 100% /snap/gnome-logs/81
/dev/loop9 135M 135M 0 100% /snap/code/25
/dev/loop10 161M 161M 0 100% /snap/gnome-3-28-1804/116
/dev/loop11 4.3M 4.3M 0 100% /snap/gnome-calculator/501
/dev/loop13 4.3M 4.3M 0 100% /snap/gnome-calculator/544
/dev/loop14 15M 15M 0 100% /snap/gnome-characters/399
/dev/loop15 90M 90M 0 100% /snap/core/7917
/dev/loop12 728M 728M 0 100% /snap/intellij-idea-ultimate/204
/dev/loop16 55M 55M 0 100% /snap/core18/1668
This CPU appears to support virtualization

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.