We have set up a HDP cluster hosted on Azure VM's. Now we want to integrate Azure AD/ Domain services to authenticate the users for accessing the cluster using their corporate credentials. How can we achieve this. ... View more

After launching hive command, it is taking more than 2-3 hours to start the Hive CLI. The screen will have below output for about 2-3 hours and then Hive CLI will start: [hdfs@hdp-m0 ~]$ hive log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender. Logging initialized using configuration in file:/etc/hive/2.6.4.5-2/0/hive-log4j.properties ... View more

I have created aHDP cluster 2.6.4.5-2 using cloudbreak 2.4.0 in Azure. I am able to successfully login to Ambari UI and most of the services are running fine, but I'm unable to open any of the quick links of all the services from Ambari UI. Below is the error I'm getting: 504 Gateway Timeout Server error - server 137.117.246.41 is unreachable at this moment. Please retry the request or contact your administrator. ... View more

I'm trying to create a new cluster with 3 nodes. I have followed all the steps given in the link: https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/ch_Getting_Ready.html but all the host registration got failed. HDPLZTESTVM is the host where I have installed ambari-server and below is the registration log for this host: ========================== Creating target directory... ========================== Command start time 2018-05-11 08:32:01 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:01 ========================== Copying ambari sudo script... ========================== Command start time 2018-05-11 08:32:01 scp /var/lib/ambari-server/ambari-sudo.sh host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying common functions script... ========================== Command start time 2018-05-11 08:32:02 scp /usr/lib/python2.6/site-packages/ambari_commons host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying create-python-wrap script... ========================== Command start time 2018-05-11 08:32:02 scp /var/lib/ambari-server/create-python-wrap.sh host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying OS type check script... ========================== Command start time 2018-05-11 08:32:02 scp /usr/lib/python2.6/site-packages/ambari_server/os_check_type.py host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Running create-python-wrap script... ========================== Command start time 2018-05-11 08:32:02 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Running OS type check... ========================== Command start time 2018-05-11 08:32:03 Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Checking 'sudo' package on remote host... ========================== Command start time 2018-05-11 08:32:03 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Copying repo file to 'tmp' folder... ========================== Command start time 2018-05-11 08:32:03 scp /etc/apt/sources.list.d/ambari.list host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Moving file to repo dir... ========================== Command start time 2018-05-11 08:32:04 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Changing permissions for ambari.repo... ========================== Command start time 2018-05-11 08:32:04 Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Update apt cache of repository... ========================== Command start time 2018-05-11 08:32:04 0% [Working] Hit:1 http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease 0% [Waiting for headers] 0% [1 InRelease gpgv 17.5 kB] [Waiting for headers] Hit:2 http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.1.0 Ambari InRelease 0% [1 InRelease gpgv 17.5 kB] 0% [Working] 0% [2 InRelease gpgv 7,394 B] 20% [Working] Reading package lists... 0% Reading package lists... 0% Reading package lists... 0% Reading package lists... 0% Reading package lists... 1% Reading package lists... 1% Reading package lists... 1% Reading package lists... 2% Reading package lists... 2% Reading package lists... Done W: http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.1.0/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1) Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Copying setup script file... ========================== Command start time 2018-05-11 08:32:04 scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:05 ========================== Running setup agent script... ========================== Command start time 2018-05-11 08:32:05 ('INFO 2018-05-11 08:32:37,604 DataCleaner.py:39 - Data cleanup thread started INFO 2018-05-11 08:32:37,606 DataCleaner.py:120 - Data cleanup started INFO 2018-05-11 08:32:37,606 DataCleaner.py:122 - Data cleanup finished INFO 2018-05-11 08:32:37,606 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZTESTVM.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,645 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2018-05-11 08:32:37,646 main.py:437 - Connecting to Ambari server at https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440 (137.xxx.xxx.xx6) INFO 2018-05-11 08:32:37,646 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/ca INFO 2018-05-11 08:32:37,714 main.py:447 - Connected to Ambari server hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net INFO 2018-05-11 08:32:37,715 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads WARNING 2018-05-11 08:32:37,715 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs. INFO 2018-05-11 08:32:37,715 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7fde6b3dde50>; currently running: False INFO 2018-05-11 08:32:37,724 hostname.py:106 - Read public hostname \'hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn() INFO 2018-05-11 08:32:37,724 Hardware.py:48 - Initializing host system information. INFO 2018-05-11 08:32:37,730 Hardware.py:176 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/1000, /run/user/0 INFO 2018-05-11 08:32:37,739 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZTESTVM.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,741 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources. INFO 2018-05-11 08:32:37,745 Hardware.py:54 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'xxxxxxxxxx.ax.internal.cloudapp.net\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.13.0-1016-azure\', \'uptime_days\': \'0\', \'memorytotal\': 16404012, \'swapfree\': \'0.00 GB\', \'memorysize\': 16404012, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': None, \'timezone\': \'UTC\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 14727340, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.13\', \'kernelversion\': \'4.13.0\', \'macaddress\': \'00:0D:3A:29:77:AA\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'137.xxx.xxx.xx6\', \'hostname\': \'hdplztestvm\', \'uptime_hours\': \'1\', \'fqdn\': \'hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'8191228\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'8191228\'}, {\'available\': \'25306356\', \'used\': \'5105908\', \'percent\': \'17%\', \'device\': \'/dev/sda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'30428648\'}, {\'available\': \'31152660\', \'used\': \'49084\', \'percent\': \'1%\', \'device\': \'/dev/sdb1\', \'mountpoint\': \'/mnt\', \'type\': \'ext4\', \'size\': \'32895792\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'6084\', \'interfaces\': \'eth0,lo\'} INFO 2018-05-11 08:32:37,865 Controller.py:170 - Registering with hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net (137.xxx.xxx.xx6) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "xxxxxxxxxx.ax.internal.cloudapp.net", "physicalprocessorcount": 4, "kernelrelease": "4.13.0-1016-azure", "uptime_days": "0", "memorytotal": 16404012, "swapfree": "0.00 GB", "memorysize": 16404012, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": null, "timezone": "UTC", "hardwareisa": "x86_64", "memoryfree": 14727340, "operatingsystem": "ubuntu", "kernelmajversion": "4.13", "kernelversion": "4.13.0", "macaddress": "00:0D:3A:29:77:AA", "operatingsystemrelease": "16.04", "ipaddress": "137.xxx.xxx.xx6", "hostname": "hdplztestvm", "uptime_hours": "1", "fqdn": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "8191228", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "8191228"}, {"available": "25306356", "used": "5105908", "percent": "17%", "device": "/dev/sda1", "mountpoint": "/", "type": "ext4", "size": "30428648"}, {"available": "31152660", "used": "49084", "percent": "1%", "device": "/dev/sdb1", "mountpoint": "/mnt", "type": "ext4", "size": "32895792"}], "hardwaremodel": "x86_64", "uptime_seconds": "6084", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.5.1.0", "agentEnv": {"transparentHugePage": "always", "hostHealth": {"agentTimeStampAtReporting": 1526027557863, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1526027557746, "hostname": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net", "responseId": -1, "publicHostname": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net"}\') INFO 2018-05-11 08:32:37,865 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/connection_info INFO 2018-05-11 08:32:37,946 security.py:93 - SSL Connect being called.. connecting to the server ', None) ('INFO 2018-05-11 08:32:37,604 DataCleaner.py:39 - Data cleanup thread started INFO 2018-05-11 08:32:37,606 DataCleaner.py:120 - Data cleanup started INFO 2018-05-11 08:32:37,606 DataCleaner.py:122 - Data cleanup finished INFO 2018-05-11 08:32:37,606 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZTESTVM.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,645 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2018-05-11 08:32:37,646 main.py:437 - Connecting to Ambari server at https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440 (137.xxx.xxx.xx6) INFO 2018-05-11 08:32:37,646 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/ca INFO 2018-05-11 08:32:37,714 main.py:447 - Connected to Ambari server hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net INFO 2018-05-11 08:32:37,715 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads WARNING 2018-05-11 08:32:37,715 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs. INFO 2018-05-11 08:32:37,715 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7fde6b3dde50>; currently running: False INFO 2018-05-11 08:32:37,724 hostname.py:106 - Read public hostname \'hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn() INFO 2018-05-11 08:32:37,724 Hardware.py:48 - Initializing host system information. INFO 2018-05-11 08:32:37,730 Hardware.py:176 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/1000, /run/user/0 INFO 2018-05-11 08:32:37,739 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZTESTVM.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,741 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources. INFO 2018-05-11 08:32:37,745 Hardware.py:54 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'xxxxxxxxxx.ax.internal.cloudapp.net\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.13.0-1016-azure\', \'uptime_days\': \'0\', \'memorytotal\': 16404012, \'swapfree\': \'0.00 GB\', \'memorysize\': 16404012, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': None, \'timezone\': \'UTC\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 14727340, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.13\', \'kernelversion\': \'4.13.0\', \'macaddress\': \'00:0D:3A:29:77:AA\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'137.xxx.xxx.xx6\', \'hostname\': \'hdplztestvm\', \'uptime_hours\': \'1\', \'fqdn\': \'hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'8191228\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'8191228\'}, {\'available\': \'25306356\', \'used\': \'5105908\', \'percent\': \'17%\', \'device\': \'/dev/sda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'30428648\'}, {\'available\': \'31152660\', \'used\': \'49084\', \'percent\': \'1%\', \'device\': \'/dev/sdb1\', \'mountpoint\': \'/mnt\', \'type\': \'ext4\', \'size\': \'32895792\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'6084\', \'interfaces\': \'eth0,lo\'} INFO 2018-05-11 08:32:37,865 Controller.py:170 - Registering with hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net (137.xxx.xxx.xx6) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "xxxxxxxxxx.ax.internal.cloudapp.net", "physicalprocessorcount": 4, "kernelrelease": "4.13.0-1016-azure", "uptime_days": "0", "memorytotal": 16404012, "swapfree": "0.00 GB", "memorysize": 16404012, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": null, "timezone": "UTC", "hardwareisa": "x86_64", "memoryfree": 14727340, "operatingsystem": "ubuntu", "kernelmajversion": "4.13", "kernelversion": "4.13.0", "macaddress": "00:0D:3A:29:77:AA", "operatingsystemrelease": "16.04", "ipaddress": "137.xxx.xxx.xx6", "hostname": "hdplztestvm", "uptime_hours": "1", "fqdn": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "8191228", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "8191228"}, {"available": "25306356", "used": "5105908", "percent": "17%", "device": "/dev/sda1", "mountpoint": "/", "type": "ext4", "size": "30428648"}, {"available": "31152660", "used": "49084", "percent": "1%", "device": "/dev/sdb1", "mountpoint": "/mnt", "type": "ext4", "size": "32895792"}], "hardwaremodel": "x86_64", "uptime_seconds": "6084", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.5.1.0", "agentEnv": {"transparentHugePage": "always", "hostHealth": {"agentTimeStampAtReporting": 1526027557863, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1526027557746, "hostname": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net", "responseId": -1, "publicHostname": "hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net"}\') INFO 2018-05-11 08:32:37,865 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/connection_info INFO 2018-05-11 08:32:37,946 security.py:93 - SSL Connect being called.. connecting to the server ', None) Connection to hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:40 Registering with the server... Registration with the server failed. HDPLZVMSN1 is one of the node where I have installed ambari-agent and below is the registration log for this host: ========================== Creating target directory... ========================== Command start time 2018-05-11 08:32:01 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:01 ========================== Copying ambari sudo script... ========================== Command start time 2018-05-11 08:32:01 scp /var/lib/ambari-server/ambari-sudo.sh host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying common functions script... ========================== Command start time 2018-05-11 08:32:02 scp /usr/lib/python2.6/site-packages/ambari_commons host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying create-python-wrap script... ========================== Command start time 2018-05-11 08:32:02 scp /var/lib/ambari-server/create-python-wrap.sh host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Copying OS type check script... ========================== Command start time 2018-05-11 08:32:02 scp /usr/lib/python2.6/site-packages/ambari_server/os_check_type.py host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:02 ========================== Running create-python-wrap script... ========================== Command start time 2018-05-11 08:32:02 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Running OS type check... ========================== Command start time 2018-05-11 08:32:03 Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Checking 'sudo' package on remote host... ========================== Command start time 2018-05-11 08:32:03 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Copying repo file to 'tmp' folder... ========================== Command start time 2018-05-11 08:32:03 scp /etc/apt/sources.list.d/ambari.list host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:03 ========================== Moving file to repo dir... ========================== Command start time 2018-05-11 08:32:03 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Changing permissions for ambari.repo... ========================== Command start time 2018-05-11 08:32:04 Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:04 ========================== Update apt cache of repository... ========================== Command start time 2018-05-11 08:32:04 0% [Working] Hit:1 http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.1.0 Ambari InRelease 0% [Working] 0% [1 InRelease gpgv 7,394 B] 20% [Working] Reading package lists... 0% Reading package lists... 0% Reading package lists... 0% Reading package lists... 0% Reading package lists... 18% Reading package lists... Done W: http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.1.0/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1) Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:05 ========================== Copying setup script file... ========================== Command start time 2018-05-11 08:32:05 scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:05 ========================== Running setup agent script... ========================== Command start time 2018-05-11 08:32:05 ('INFO 2018-05-11 08:32:37,836 DataCleaner.py:39 - Data cleanup thread started INFO 2018-05-11 08:32:37,837 DataCleaner.py:120 - Data cleanup started INFO 2018-05-11 08:32:37,837 DataCleaner.py:122 - Data cleanup finished INFO 2018-05-11 08:32:37,838 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZVMSN1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,861 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2018-05-11 08:32:37,864 main.py:437 - Connecting to Ambari server at https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440 (137.xxx.xxx.xx6) INFO 2018-05-11 08:32:37,864 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/ca INFO 2018-05-11 08:32:37,950 main.py:447 - Connected to Ambari server hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net INFO 2018-05-11 08:32:37,951 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads WARNING 2018-05-11 08:32:37,952 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs. INFO 2018-05-11 08:32:37,952 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f056e28bdd0>; currently running: False INFO 2018-05-11 08:32:37,965 hostname.py:106 - Read public hostname \'hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn() INFO 2018-05-11 08:32:37,966 Hardware.py:48 - Initializing host system information. INFO 2018-05-11 08:32:37,973 Hardware.py:176 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0, /run/user/1000 INFO 2018-05-11 08:32:37,983 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZVMSN1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,985 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources. INFO 2018-05-11 08:32:37,989 Hardware.py:54 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'xxxxxxxxxx.ax.internal.cloudapp.net\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.13.0-1014-azure\', \'uptime_days\': \'1\', \'memorytotal\': 16404012, \'swapfree\': \'0.00 GB\', \'memorysize\': 16404012, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': None, \'timezone\': \'UTC\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 14937452, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.13\', \'kernelversion\': \'4.13.0\', \'macaddress\': \'00:0D:3A:29:6C:6A\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'137.xxx.xxx.xx0\', \'hostname\': \'hdplzvmsn1\', \'uptime_hours\': \'27\', \'fqdn\': \'hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'8191228\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'8191228\'}, {\'available\': \'28704176\', \'used\': \'1708088\', \'percent\': \'6%\', \'device\': \'/dev/sda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'30428648\'}, {\'available\': \'31152660\', \'used\': \'49084\', \'percent\': \'1%\', \'device\': \'/dev/sdb1\', \'mountpoint\': \'/mnt\', \'type\': \'ext4\', \'size\': \'32895792\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'99344\', \'interfaces\': \'eth0,lo\'} INFO 2018-05-11 08:32:38,121 Controller.py:170 - Registering with hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net (137.xxx.xxx.xx0) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "xxxxxxxxxx.ax.internal.cloudapp.net", "physicalprocessorcount": 4, "kernelrelease": "4.13.0-1014-azure", "uptime_days": "1", "memorytotal": 16404012, "swapfree": "0.00 GB", "memorysize": 16404012, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": null, "timezone": "UTC", "hardwareisa": "x86_64", "memoryfree": 14937452, "operatingsystem": "ubuntu", "kernelmajversion": "4.13", "kernelversion": "4.13.0", "macaddress": "00:0D:3A:29:6C:6A", "operatingsystemrelease": "16.04", "ipaddress": "137.xxx.xxx.xx0", "hostname": "hdplzvmsn1", "uptime_hours": "27", "fqdn": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "8191228", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "8191228"}, {"available": "28704176", "used": "1708088", "percent": "6%", "device": "/dev/sda1", "mountpoint": "/", "type": "ext4", "size": "30428648"}, {"available": "31152660", "used": "49084", "percent": "1%", "device": "/dev/sdb1", "mountpoint": "/mnt", "type": "ext4", "size": "32895792"}], "hardwaremodel": "x86_64", "uptime_seconds": "99344", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.5.1.0", "agentEnv": {"transparentHugePage": "always", "hostHealth": {"agentTimeStampAtReporting": 1526027558120, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1526027557990, "hostname": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net", "responseId": -1, "publicHostname": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net"}\') INFO 2018-05-11 08:32:38,122 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/connection_info INFO 2018-05-11 08:32:38,181 security.py:93 - SSL Connect being called.. connecting to the server ', None) ('INFO 2018-05-11 08:32:37,836 DataCleaner.py:39 - Data cleanup thread started INFO 2018-05-11 08:32:37,837 DataCleaner.py:120 - Data cleanup started INFO 2018-05-11 08:32:37,837 DataCleaner.py:122 - Data cleanup finished INFO 2018-05-11 08:32:37,838 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZVMSN1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,861 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2018-05-11 08:32:37,864 main.py:437 - Connecting to Ambari server at https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440 (137.xxx.xxx.xx6) INFO 2018-05-11 08:32:37,864 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/ca INFO 2018-05-11 08:32:37,950 main.py:447 - Connected to Ambari server hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net INFO 2018-05-11 08:32:37,951 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads WARNING 2018-05-11 08:32:37,952 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs. INFO 2018-05-11 08:32:37,952 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f056e28bdd0>; currently running: False INFO 2018-05-11 08:32:37,965 hostname.py:106 - Read public hostname \'hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn() INFO 2018-05-11 08:32:37,966 Hardware.py:48 - Initializing host system information. INFO 2018-05-11 08:32:37,973 Hardware.py:176 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0, /run/user/1000 INFO 2018-05-11 08:32:37,983 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'HDPLZVMSN1.xxxxxxxxxx.ax.internal.cloudapp.net\' using socket.getfqdn(). INFO 2018-05-11 08:32:37,985 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources. INFO 2018-05-11 08:32:37,989 Hardware.py:54 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'xxxxxxxxxx.ax.internal.cloudapp.net\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.13.0-1014-azure\', \'uptime_days\': \'1\', \'memorytotal\': 16404012, \'swapfree\': \'0.00 GB\', \'memorysize\': 16404012, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': None, \'timezone\': \'UTC\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 14937452, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.13\', \'kernelversion\': \'4.13.0\', \'macaddress\': \'00:0D:3A:29:6C:6A\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'137.xxx.xxx.xx0\', \'hostname\': \'hdplzvmsn1\', \'uptime_hours\': \'27\', \'fqdn\': \'hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'8191228\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'8191228\'}, {\'available\': \'28704176\', \'used\': \'1708088\', \'percent\': \'6%\', \'device\': \'/dev/sda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'30428648\'}, {\'available\': \'31152660\', \'used\': \'49084\', \'percent\': \'1%\', \'device\': \'/dev/sdb1\', \'mountpoint\': \'/mnt\', \'type\': \'ext4\', \'size\': \'32895792\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'99344\', \'interfaces\': \'eth0,lo\'} INFO 2018-05-11 08:32:38,121 Controller.py:170 - Registering with hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net (137.xxx.xxx.xx0) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "xxxxxxxxxx.ax.internal.cloudapp.net", "physicalprocessorcount": 4, "kernelrelease": "4.13.0-1014-azure", "uptime_days": "1", "memorytotal": 16404012, "swapfree": "0.00 GB", "memorysize": 16404012, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": null, "timezone": "UTC", "hardwareisa": "x86_64", "memoryfree": 14937452, "operatingsystem": "ubuntu", "kernelmajversion": "4.13", "kernelversion": "4.13.0", "macaddress": "00:0D:3A:29:6C:6A", "operatingsystemrelease": "16.04", "ipaddress": "137.xxx.xxx.xx0", "hostname": "hdplzvmsn1", "uptime_hours": "27", "fqdn": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "8191228", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "8191228"}, {"available": "28704176", "used": "1708088", "percent": "6%", "device": "/dev/sda1", "mountpoint": "/", "type": "ext4", "size": "30428648"}, {"available": "31152660", "used": "49084", "percent": "1%", "device": "/dev/sdb1", "mountpoint": "/mnt", "type": "ext4", "size": "32895792"}], "hardwaremodel": "x86_64", "uptime_seconds": "99344", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.5.1.0", "agentEnv": {"transparentHugePage": "always", "hostHealth": {"agentTimeStampAtReporting": 1526027558120, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1526027557990, "hostname": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net", "responseId": -1, "publicHostname": "hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net"}\') INFO 2018-05-11 08:32:38,122 NetUtil.py:70 - Connecting to https://hdplztestvm.xxxxxxxxxx.ax.internal.cloudapp.net:8440/connection_info INFO 2018-05-11 08:32:38,181 security.py:93 - SSL Connect being called.. connecting to the server ', None) Connection to hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net closed. SSH command execution finished host=hdplzvmsn1.xxxxxxxxxx.ax.internal.cloudapp.net, exitcode=0 Command end time 2018-05-11 08:32:40 Registering with the server... Registration with the server failed. I'm unable to get the exact issue from these logs. Please help me to proceed further. ... View more