INFO 2016-12-21 11:37:33,984 ActionQueue.py:104 - Adding STATUS_COMMAND for component INFRA_SOLR of service AMBARI_INFRA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:33,993 ActionQueue.py:104 - Adding STATUS_COMMAND for component INFRA_SOLR_CLIENT of service AMBARI_INFRA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,001 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_MONITOR of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,010 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_COLLECTOR of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,105 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_GRAFANA of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,127 ActionQueue.py:104 - Adding STATUS_COMMAND for component ATLAS_CLIENT of service ATLAS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,210 ActionQueue.py:104 - Adding STATUS_COMMAND for component NAMENODE of service HDFS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,300 ActionQueue.py:104 - Adding STATUS_COMMAND for component HCAT of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,314 ActionQueue.py:104 - Adding STATUS_COMMAND for component WEBHCAT_SERVER of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,336 ActionQueue.py:104 - Adding STATUS_COMMAND for component KAFKA_BROKER of service KAFKA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,403 ActionQueue.py:104 - Adding STATUS_COMMAND for component OOZIE_CLIENT of service OOZIE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,417 ActionQueue.py:104 - Adding STATUS_COMMAND for component PIG of service PIG of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,431 ActionQueue.py:104 - Adding STATUS_COMMAND for component RANGER_ADMIN of service RANGER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,533 ActionQueue.py:104 - Adding STATUS_COMMAND for component NIMBUS of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,555 ActionQueue.py:104 - Adding STATUS_COMMAND for component DRPC_SERVER of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:34,698 ActionQueue.py:104 - Adding STATUS_COMMAND for component ZOOKEEPER_CLIENT of service ZOOKEEPER of cluster Sandbox to the queue. DEBUG 2016-12-21 11:37:38,937 script_alert.py:155 - [Alert][namenode_ha_health] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_ha_namenode_health.py DEBUG 2016-12-21 11:37:38,937 script_alert.py:155 - [Alert][namenode_ha_health] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_ha_namenode_health.py INFO 2016-12-21 11:37:38,945 logger.py:71 - checked_call['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4 -kt /etc/security/keytabs/spnego.service.keytab HTTP/sandbox.hortonworks.com@HORTONWORKS.COM > /dev/null'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:38,945 logger.py:71 - checked_call['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4 -kt /etc/security/keytabs/spnego.service.keytab HTTP/sandbox.hortonworks.com@HORTONWORKS.COM > /dev/null'] {'user': 'ambari-qa'} DEBUG 2016-12-21 11:37:38,952 script_alert.py:155 - [Alert][upgrade_finalized_state] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_upgrade_finalized.py DEBUG 2016-12-21 11:37:38,952 script_alert.py:155 - [Alert][upgrade_finalized_state] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_upgrade_finalized.py DEBUG 2016-12-21 11:37:38,956 script_alert.py:155 - [Alert][namenode_last_checkpoint] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py DEBUG 2016-12-21 11:37:38,956 script_alert.py:155 - [Alert][namenode_last_checkpoint] Executing script check /var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py DEBUG 2016-12-21 11:37:38,961 script_alert.py:155 - [Alert][zeppelin_server_status] Executing script check /var/lib/ambari-agent/cache/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/alert_check_zeppelin.py DEBUG 2016-12-21 11:37:38,961 script_alert.py:155 - [Alert][zeppelin_server_status] Executing script check /var/lib/ambari-agent/cache/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/alert_check_zeppelin.py DEBUG 2016-12-21 11:37:38,963 script_alert.py:103 - The script does not have a get_tokens() function DEBUG 2016-12-21 11:37:38,963 script_alert.py:103 - The script does not have a get_tokens() function DEBUG 2016-12-21 11:37:38,965 script_alert.py:155 - [Alert][yarn_nodemanager_health] Executing script check /var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py ERROR 2016-12-21 11:37:38,965 script_alert.py:119 - [Alert][zeppelin_server_status] Failed with result CRITICAL: ['Zeppelin is not running'] DEBUG 2016-12-21 11:37:38,965 script_alert.py:155 - [Alert][yarn_nodemanager_health] Executing script check /var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py ERROR 2016-12-21 11:37:38,965 script_alert.py:119 - [Alert][zeppelin_server_status] Failed with result CRITICAL: ['Zeppelin is not running'] DEBUG 2016-12-21 11:37:38,969 script_alert.py:155 - [Alert][nodemanager_health_summary] Executing script check /var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanagers_summary.py DEBUG 2016-12-21 11:37:38,969 script_alert.py:155 - [Alert][nodemanager_health_summary] Executing script check /var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanagers_summary.py DEBUG 2016-12-21 11:37:38,995 script_alert.py:155 - [Alert][hive_webhcat_server_status] Executing script check /var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_webhcat_server.py DEBUG 2016-12-21 11:37:38,995 script_alert.py:155 - [Alert][hive_webhcat_server_status] Executing script check /var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_webhcat_server.py DEBUG 2016-12-21 11:37:39,001 script_alert.py:155 - [Alert][ams_metrics_monitor_process] Executing script check /var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/alerts/alert_ambari_metrics_monitor.py DEBUG 2016-12-21 11:37:39,001 script_alert.py:155 - [Alert][ams_metrics_monitor_process] Executing script check /var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/alerts/alert_ambari_metrics_monitor.py INFO 2016-12-21 11:37:39,004 logger.py:71 - checked_call returned (0, '') INFO 2016-12-21 11:37:39,004 logger.py:71 - checked_call returned (0, '') INFO 2016-12-21 11:37:39,004 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/upgrade_finalized_state_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,004 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/upgrade_finalized_state_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,006 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/191dfe54-5d6e-4c34-876a-a45cb578924b -c /var/lib/ambari-agent/tmp/cookies/191dfe54-5d6e-4c34-876a-a45cb578924b '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmp3x460t 2>/tmp/tmpTYKh4l''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,006 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/191dfe54-5d6e-4c34-876a-a45cb578924b -c /var/lib/ambari-agent/tmp/cookies/191dfe54-5d6e-4c34-876a-a45cb578924b '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmp3x460t 2>/tmp/tmpTYKh4l''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} DEBUG 2016-12-21 11:37:39,019 script_alert.py:155 - [Alert][ambari_agent_disk_usage] Executing script check /var/lib/ambari-agent/cache/host_scripts/alert_disk_space.py DEBUG 2016-12-21 11:37:39,019 script_alert.py:155 - [Alert][ambari_agent_disk_usage] Executing script check /var/lib/ambari-agent/cache/host_scripts/alert_disk_space.py INFO 2016-12-21 11:37:39,025 logger.py:71 - Pid file /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid is empty or does not exist INFO 2016-12-21 11:37:39,025 logger.py:71 - Pid file /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid is empty or does not exist ERROR 2016-12-21 11:37:39,025 script_alert.py:119 - [Alert][ams_metrics_monitor_process] Failed with result CRITICAL: ['Ambari Monitor is NOT running on sandbox.hortonworks.com'] ERROR 2016-12-21 11:37:39,025 script_alert.py:119 - [Alert][ams_metrics_monitor_process] Failed with result CRITICAL: ['Ambari Monitor is NOT running on sandbox.hortonworks.com'] INFO 2016-12-21 11:37:39,055 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,055 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,055 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,055 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,059 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/1c24523b-6a98-4287-aaec-b7208065985e -c /var/lib/ambari-agent/tmp/cookies/1c24523b-6a98-4287-aaec-b7208065985e '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 10 --max-time 12 1>/tmp/tmp2VvMOI 2>/tmp/tmpDn1AH3''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/upgrade_finalized_state_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,059 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/1c24523b-6a98-4287-aaec-b7208065985e -c /var/lib/ambari-agent/tmp/cookies/1c24523b-6a98-4287-aaec-b7208065985e '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 10 --max-time 12 1>/tmp/tmp2VvMOI 2>/tmp/tmpDn1AH3''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/upgrade_finalized_state_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,065 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,065 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,101 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,101 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,102 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,102 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,105 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/261b8ba4-a30f-4abe-bbb8-c5a1b08cd2ac -c /var/lib/ambari-agent/tmp/cookies/261b8ba4-a30f-4abe-bbb8-c5a1b08cd2ac '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpLK_sFm 2>/tmp/tmpielpwq''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,105 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/261b8ba4-a30f-4abe-bbb8-c5a1b08cd2ac -c /var/lib/ambari-agent/tmp/cookies/261b8ba4-a30f-4abe-bbb8-c5a1b08cd2ac '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpLK_sFm 2>/tmp/tmpielpwq''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,111 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,111 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,150 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,150 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,150 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,150 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,154 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/d7fbe285-2c86-4a56-b8a4-f56b936994ad -c /var/lib/ambari-agent/tmp/cookies/d7fbe285-2c86-4a56-b8a4-f56b936994ad '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpgNxuc6 2>/tmp/tmpvgWgx6''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,154 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/d7fbe285-2c86-4a56-b8a4-f56b936994ad -c /var/lib/ambari-agent/tmp/cookies/d7fbe285-2c86-4a56-b8a4-f56b936994ad '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpgNxuc6 2>/tmp/tmpvgWgx6''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,158 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,158 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,197 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,197 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,198 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_summary_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,198 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_summary_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,201 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 -c /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 https://sandbox.hortonworks.com:9997/ws/v1/node/info --connect-timeout 5 --max-time 7 1>/tmp/tmp1bXZ3m 2>/tmp/tmp33I0ee''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,201 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 -c /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 https://sandbox.hortonworks.com:9997/ws/v1/node/info --connect-timeout 5 --max-time 7 1>/tmp/tmp1bXZ3m 2>/tmp/tmp33I0ee''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,207 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,207 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,243 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,243 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,243 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,243 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,247 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e -c /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e '"'"'https://sandbox.hortonworks.com:1544/jmx?qry=Hadoop:service=ResourceManager,name=RMNMInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmp2XeeIW 2>/tmp/tmpDxT6on''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_summary_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,247 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e -c /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e '"'"'https://sandbox.hortonworks.com:1544/jmx?qry=Hadoop:service=ResourceManager,name=RMNMInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmp2XeeIW 2>/tmp/tmpDxT6on''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/nm_health_summary_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,253 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,253 logger.py:71 - call returned (7, '') ERROR 2016-12-21 11:37:39,254 script_alert.py:119 - [Alert][yarn_nodemanager_health] Failed with result CRITICAL: ['Connection failed to https://sandbox.hortonworks.com:9997/ws/v1/node/info (Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py", line 166, in execute\n connection_timeout=curl_connection_timeout, kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 198, in curl_krb_request\n _, curl_stdout, curl_stderr = get_user_call_output(curl_command, user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 -c /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 https://sandbox.hortonworks.com:9997/ws/v1/node/info --connect-timeout 5 --max-time 7 1>/tmp/tmp1bXZ3m 2>/tmp/tmp33I0ee\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:9997; Connection refused\n\n)'] ERROR 2016-12-21 11:37:39,254 script_alert.py:119 - [Alert][yarn_nodemanager_health] Failed with result CRITICAL: ['Connection failed to https://sandbox.hortonworks.com:9997/ws/v1/node/info (Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py", line 166, in execute\n connection_timeout=curl_connection_timeout, kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 198, in curl_krb_request\n _, curl_stdout, curl_stderr = get_user_call_output(curl_command, user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 -c /var/lib/ambari-agent/tmp/cookies/e5c2256a-f200-4ee7-ad6c-e837dee37d80 https://sandbox.hortonworks.com:9997/ws/v1/node/info --connect-timeout 5 --max-time 7 1>/tmp/tmp1bXZ3m 2>/tmp/tmp33I0ee\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:9997; Connection refused\n\n)'] INFO 2016-12-21 11:37:39,289 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,289 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,289 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,289 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,293 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/90a4c3c5-af70-4cf1-a0cd-80d2c2d4ce73 -c /var/lib/ambari-agent/tmp/cookies/90a4c3c5-af70-4cf1-a0cd-80d2c2d4ce73 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:9200 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpYaYUO8 2>/tmp/tmp25eASu''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,293 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/90a4c3c5-af70-4cf1-a0cd-80d2c2d4ce73 -c /var/lib/ambari-agent/tmp/cookies/90a4c3c5-af70-4cf1-a0cd-80d2c2d4ce73 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:9200 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpYaYUO8 2>/tmp/tmp25eASu''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,296 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,296 logger.py:71 - call returned (7, '') DEBUG 2016-12-21 11:37:39,298 script_alert.py:121 - [Alert][nodemanager_health_summary] Failed with result UNKNOWN: ['Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanagers_summary.py", line 138, in execute\n kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 198, in curl_krb_request\n _, curl_stdout, curl_stderr = get_user_call_output(curl_command, user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e -c /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e \'https://sandbox.hortonworks.com:1544/jmx?qry=Hadoop:service=ResourceManager,name=RMNMInfo\' --connect-timeout 5 --max-time 7 1>/tmp/tmp2XeeIW 2>/tmp/tmpDxT6on\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:1544; Connection refused\n\n'] DEBUG 2016-12-21 11:37:39,298 script_alert.py:121 - [Alert][nodemanager_health_summary] Failed with result UNKNOWN: ['Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanagers_summary.py", line 138, in execute\n kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 198, in curl_krb_request\n _, curl_stdout, curl_stderr = get_user_call_output(curl_command, user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e -c /var/lib/ambari-agent/tmp/cookies/f142e85f-8c49-446c-bdcc-8faaae17a94e \'https://sandbox.hortonworks.com:1544/jmx?qry=Hadoop:service=ResourceManager,name=RMNMInfo\' --connect-timeout 5 --max-time 7 1>/tmp/tmp2XeeIW 2>/tmp/tmpDxT6on\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:1544; Connection refused\n\n'] INFO 2016-12-21 11:37:39,334 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,334 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,334 logger.py:71 - checked_call['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/curl_krb_cache/webhcat_alert_cc__ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0 -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa-Sandbox@HORTONWORKS.COM > /dev/null'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,334 logger.py:71 - checked_call['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/curl_krb_cache/webhcat_alert_cc__ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0 -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa-Sandbox@HORTONWORKS.COM > /dev/null'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,336 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e191bd36-8fdf-492b-86d0-0a82373689dc -c /var/lib/ambari-agent/tmp/cookies/e191bd36-8fdf-492b-86d0-0a82373689dc -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:1558 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpSKQbtl 2>/tmp/tmpPXbwxC''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,336 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/e191bd36-8fdf-492b-86d0-0a82373689dc -c /var/lib/ambari-agent/tmp/cookies/e191bd36-8fdf-492b-86d0-0a82373689dc -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:1558 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpSKQbtl 2>/tmp/tmpPXbwxC''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,345 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,345 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,383 logger.py:71 - checked_call returned (0, '') INFO 2016-12-21 11:37:39,383 logger.py:71 - checked_call returned (0, '') INFO 2016-12-21 11:37:39,383 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,383 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,386 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -c /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -w '"'"'%{http_code}'"'"' '"'"'http://sandbox.hortonworks.com:50111/templeton/v1/status?user.name=ambari-qa'"'"' --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpMd2KD9 2>/tmp/tmp4KnBZ0''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/webhcat_alert_cc__ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,386 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -c /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -w '"'"'%{http_code}'"'"' '"'"'http://sandbox.hortonworks.com:50111/templeton/v1/status?user.name=ambari-qa'"'"' --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpMd2KD9 2>/tmp/tmp4KnBZ0''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/webhcat_alert_cc__ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,391 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,391 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,428 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,428 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,428 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,428 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,429 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/40af1bd9-efe4-4e0d-83a6-d470652e22c0 -c /var/lib/ambari-agent/tmp/cookies/40af1bd9-efe4-4e0d-83a6-d470652e22c0 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:50070 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpmZRaFb 2>/tmp/tmppb9sNG''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,429 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/40af1bd9-efe4-4e0d-83a6-d470652e22c0 -c /var/lib/ambari-agent/tmp/cookies/40af1bd9-efe4-4e0d-83a6-d470652e22c0 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:50070 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpmZRaFb 2>/tmp/tmppb9sNG''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,437 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,437 logger.py:71 - call returned (7, '') ERROR 2016-12-21 11:37:39,438 script_alert.py:119 - [Alert][hive_webhcat_server_status] Failed with result CRITICAL: ['Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_webhcat_server.py", line 160, in execute\n kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 181, in curl_krb_request\n user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -c /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -w \'%{http_code}\' \'http://sandbox.hortonworks.com:50111/templeton/v1/status?user.name=ambari-qa\' --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpMd2KD9 2>/tmp/tmp4KnBZ0\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:50111; Connection refused\n000\n'] ERROR 2016-12-21 11:37:39,438 script_alert.py:119 - [Alert][hive_webhcat_server_status] Failed with result CRITICAL: ['Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_webhcat_server.py", line 160, in execute\n kinit_timer_ms = kinit_timer_ms)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/curl_krb_request.py", line 181, in curl_krb_request\n user=user, env=kerberos_env)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output\n raise Fail(err_msg)\nFail: Execution of \'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -c /var/lib/ambari-agent/tmp/cookies/bfea76f1-07b0-4c7f-adc4-e1506b0e390b -w \'%{http_code}\' \'http://sandbox.hortonworks.com:50111/templeton/v1/status?user.name=ambari-qa\' --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpMd2KD9 2>/tmp/tmp4KnBZ0\' returned 7. % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to sandbox.hortonworks.com:50111; Connection refused\n000\n'] INFO 2016-12-21 11:37:39,474 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,474 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,474 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,474 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,479 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/85d8a63d-25d0-4960-80d9-2b0cc0c18ff8 -c /var/lib/ambari-agent/tmp/cookies/85d8a63d-25d0-4960-80d9-2b0cc0c18ff8 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:50090 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpBv4Nlq 2>/tmp/tmp0S818U''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,479 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/85d8a63d-25d0-4960-80d9-2b0cc0c18ff8 -c /var/lib/ambari-agent/tmp/cookies/85d8a63d-25d0-4960-80d9-2b0cc0c18ff8 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:50090 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpBv4Nlq 2>/tmp/tmp0S818U''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,485 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,485 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,521 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,521 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,521 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,521 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,525 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/420e2cb5-e2bf-4269-ab68-a89c9159cc56 -c /var/lib/ambari-agent/tmp/cookies/420e2cb5-e2bf-4269-ab68-a89c9159cc56 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:8190/ws/v1/timeline --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpsNh3BF 2>/tmp/tmph7bJ6Y''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,525 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/420e2cb5-e2bf-4269-ab68-a89c9159cc56 -c /var/lib/ambari-agent/tmp/cookies/420e2cb5-e2bf-4269-ab68-a89c9159cc56 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:8190/ws/v1/timeline --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpsNh3BF 2>/tmp/tmph7bJ6Y''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,530 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,530 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,566 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,566 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,566 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,566 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,569 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/847e4f94-2e72-4dec-af7c-8830848f19b5 -c /var/lib/ambari-agent/tmp/cookies/847e4f94-2e72-4dec-af7c-8830848f19b5 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:9997 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpfnKbMs 2>/tmp/tmpNGrPRO''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,569 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/847e4f94-2e72-4dec-af7c-8830848f19b5 -c /var/lib/ambari-agent/tmp/cookies/847e4f94-2e72-4dec-af7c-8830848f19b5 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:9997 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpfnKbMs 2>/tmp/tmpNGrPRO''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,575 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,575 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,611 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,611 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,611 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,611 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,612 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/a69de4a7-8def-4f56-81b0-cb92112e08a9 -c /var/lib/ambari-agent/tmp/cookies/a69de4a7-8def-4f56-81b0-cb92112e08a9 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:8744 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpNerG65 2>/tmp/tmpbdAiNw''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,612 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/a69de4a7-8def-4f56-81b0-cb92112e08a9 -c /var/lib/ambari-agent/tmp/cookies/a69de4a7-8def-4f56-81b0-cb92112e08a9 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:8744 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpNerG65 2>/tmp/tmpbdAiNw''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,621 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,621 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,656 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,656 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,657 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,657 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,660 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/a28e0b06-28ab-416f-81f1-eb51b3e02c1f -c /var/lib/ambari-agent/tmp/cookies/a28e0b06-28ab-416f-81f1-eb51b3e02c1f -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:1544 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmp8Sn6op 2>/tmp/tmpGmKgPo''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,660 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/a28e0b06-28ab-416f-81f1-eb51b3e02c1f -c /var/lib/ambari-agent/tmp/cookies/a28e0b06-28ab-416f-81f1-eb51b3e02c1f -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:1544 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmp8Sn6op 2>/tmp/tmpGmKgPo''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,665 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,665 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,701 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,701 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,701 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,701 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,705 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/548aec31-8b9f-4d43-a698-119562680f58 -c /var/lib/ambari-agent/tmp/cookies/548aec31-8b9f-4d43-a698-119562680f58 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:1551/login.jsp --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpUw94lt 2>/tmp/tmphm9XrT''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,705 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/548aec31-8b9f-4d43-a698-119562680f58 -c /var/lib/ambari-agent/tmp/cookies/548aec31-8b9f-4d43-a698-119562680f58 -w '"'"'%{http_code}'"'"' https://sandbox.hortonworks.com:1551/login.jsp --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpUw94lt 2>/tmp/tmphm9XrT''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,711 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,711 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,746 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,746 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,746 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,746 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,751 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/09f8cf9d-ad0d-458f-b029-fa3ece50ff69 -c /var/lib/ambari-agent/tmp/cookies/09f8cf9d-ad0d-458f-b029-fa3ece50ff69 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:1554 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpCbzXxx 2>/tmp/tmpfp9g_g''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,751 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/09f8cf9d-ad0d-458f-b029-fa3ece50ff69 -c /var/lib/ambari-agent/tmp/cookies/09f8cf9d-ad0d-458f-b029-fa3ece50ff69 -w '"'"'%{http_code}'"'"' http://sandbox.hortonworks.com:1554 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpCbzXxx 2>/tmp/tmpfp9g_g''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,791 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,791 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,791 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,791 logger.py:71 - call['/usr/bin/klist -s /var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'] {'user': 'ambari-qa'} INFO 2016-12-21 11:37:39,796 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/76a54eab-ab84-4e5e-9236-55a98d795516 -c /var/lib/ambari-agent/tmp/cookies/76a54eab-ab84-4e5e-9236-55a98d795516 -w '"'"'%{http_code}'"'"' http://localhost:1549/api/atlas/admin/status --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpTRT_Yv 2>/tmp/tmpdWItQp''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,796 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/76a54eab-ab84-4e5e-9236-55a98d795516 -c /var/lib/ambari-agent/tmp/cookies/76a54eab-ab84-4e5e-9236-55a98d795516 -w '"'"'%{http_code}'"'"' http://localhost:1549/api/atlas/admin/status --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpTRT_Yv 2>/tmp/tmpdWItQp''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/web_alert_ambari-qa_cc_08dfc02dd342d30ad7f60ce9de4871b0'}} INFO 2016-12-21 11:37:39,801 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,801 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,838 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,838 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,838 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/8c5a6db4-c8c1-4f27-a87d-be6dde8cf8da -c /var/lib/ambari-agent/tmp/cookies/8c5a6db4-c8c1-4f27-a87d-be6dde8cf8da '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpIyqiTN 2>/tmp/tmpTCTgYJ''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,838 logger.py:71 - call['ambari-sudo.sh su ambari-qa -l -s /bin/bash -c 'curl -L -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/8c5a6db4-c8c1-4f27-a87d-be6dde8cf8da -c /var/lib/ambari-agent/tmp/cookies/8c5a6db4-c8c1-4f27-a87d-be6dde8cf8da '"'"'http://sandbox.hortonworks.com:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'"'"' --connect-timeout 5 --max-time 7 1>/tmp/tmpIyqiTN 2>/tmp/tmpTCTgYJ''] {'quiet': False, 'env': {'KRB5CCNAME': '/var/lib/ambari-agent/tmp/curl_krb_cache/checkpoint_time_alert_ambari-qa_cc_2457ccdef80ee5c534798e1b3124feb4'}} INFO 2016-12-21 11:37:39,847 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,847 logger.py:71 - call returned (7, '') INFO 2016-12-21 11:37:39,869 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,869 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,889 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:39,889 logger.py:71 - call returned (0, '') DEBUG 2016-12-21 11:37:39,891 alert_checkpoint_time.py:224 - Raising critical alert: transaction_difference = 59, checkpoint_tx = 1000000 DEBUG 2016-12-21 11:37:39,891 alert_checkpoint_time.py:224 - Raising critical alert: transaction_difference = 59, checkpoint_tx = 1000000 ERROR 2016-12-21 11:37:39,891 script_alert.py:119 - [Alert][namenode_last_checkpoint] Failed with result CRITICAL: ['Last Checkpoint: [50 hours, 1 minutes, 59 transactions]'] ERROR 2016-12-21 11:37:39,891 script_alert.py:119 - [Alert][namenode_last_checkpoint] Failed with result CRITICAL: ['Last Checkpoint: [50 hours, 1 minutes, 59 transactions]'] INFO 2016-12-21 11:37:52,238 Controller.py:277 - Heartbeat with server is running... INFO 2016-12-21 11:37:52,240 Heartbeat.py:90 - Adding host info/state to heartbeat message. INFO 2016-12-21 11:37:52,286 logger.py:71 - call[['test', '-w', '/']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,286 logger.py:71 - call[['test', '-w', '/']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,293 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,293 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,293 logger.py:71 - call[['test', '-w', '/dev']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,293 logger.py:71 - call[['test', '-w', '/dev']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,300 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,300 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,300 logger.py:71 - call[['test', '-w', '/dev/shm']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,300 logger.py:71 - call[['test', '-w', '/dev/shm']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,306 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,306 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,306 logger.py:71 - call[['test', '-w', '/run']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,306 logger.py:71 - call[['test', '-w', '/run']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,312 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,312 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,313 logger.py:71 - call[['test', '-w', '/sys/fs/cgroup']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,313 logger.py:71 - call[['test', '-w', '/sys/fs/cgroup']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,319 logger.py:71 - call returned (1, '') INFO 2016-12-21 11:37:52,319 logger.py:71 - call returned (1, '') INFO 2016-12-21 11:37:52,319 logger.py:71 - call[['test', '-w', '/mnt']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,319 logger.py:71 - call[['test', '-w', '/mnt']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,326 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,326 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,326 logger.py:71 - call[['test', '-w', '/run/user/1023']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,326 logger.py:71 - call[['test', '-w', '/run/user/1023']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,332 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,332 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,332 logger.py:71 - call[['test', '-w', '/run/user/2606233']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,332 logger.py:71 - call[['test', '-w', '/run/user/2606233']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:52,338 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:52,338 logger.py:71 - call returned (0, '') ERROR 2016-12-21 11:37:53,247 Controller.py:341 - Error in responseId sequence - restarting INFO 2016-12-21 11:37:53,248 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-12-21 11:37:53,248 threadpool.py:111 - Shutting down thread pool INFO 2016-12-21 11:37:53,275 scheduler.py:606 - Scheduler has been shut down INFO 2016-12-21 11:37:53,276 threadpool.py:52 - Started thread pool with 3 core threads and 20 maximum threads INFO 2016-12-21 11:37:53,277 AlertSchedulerHandler.py:166 - [AlertScheduler] Stopped the alert scheduler. INFO 2016-12-21 11:37:53,277 threadpool.py:111 - Shutting down thread pool INFO 2016-12-21 11:37:53,277 ExitHelper.py:67 - Cleanup finished, exiting with code:77 INFO 2016-12-21 11:37:53,417 main.py:90 - loglevel=logging.DEBUG INFO 2016-12-21 11:37:53,417 main.py:90 - loglevel=logging.DEBUG INFO 2016-12-21 11:37:53,417 main.py:90 - loglevel=logging.DEBUG INFO 2016-12-21 11:37:53,418 DataCleaner.py:39 - Data cleanup thread started INFO 2016-12-21 11:37:53,419 DataCleaner.py:120 - Data cleanup started DEBUG 2016-12-21 11:37:53,420 DataCleaner.py:71 - Cleaning up inside directory /var/lib/ambari-agent/data INFO 2016-12-21 11:37:53,423 DataCleaner.py:122 - Data cleanup finished INFO 2016-12-21 11:37:53,489 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2016-12-21 11:37:53,491 main.py:349 - Connecting to Ambari server at https://sandbox.hortonworks.com:8440 (10.69.160.144) INFO 2016-12-21 11:37:53,493 NetUtil.py:62 - Connecting to https://sandbox.hortonworks.com:8440/ca INFO 2016-12-21 11:37:53,545 main.py:359 - Connected to Ambari server sandbox.hortonworks.com WARNING 2016-12-21 11:37:53,582 ClusterConfiguration.py:71 - Unable to load configurations from /var/lib/ambari-agent/cache/cluster_configuration/configurations.json. This file will be regenerated on registration INFO 2016-12-21 11:37:53,583 threadpool.py:52 - Started thread pool with 3 core threads and 20 maximum threads INFO 2016-12-21 11:37:53,600 AlertSchedulerHandler.py:271 - [AlertScheduler] Caching cluster Sandbox with alert hash 1c065afe076bfd3cd6cd428f3e9331a6 INFO 2016-12-21 11:37:53,619 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,620 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling kafka_broker_process with UUID 4cc83d4f-00bc-4f28-884b-bbaa8a903632 INFO 2016-12-21 11:37:53,620 AlertSchedulerHandler.py:338 - [AlertScheduler] The alert mapreduce_history_server_process with UUID d7fec1e4-852d-44ae-8204-5630c4e07872 is disabled and will not be scheduled INFO 2016-12-21 11:37:53,620 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,620 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling mapreduce_history_server_webui with UUID 0ab39ec6-6707-4603-91a2-d7444346920e INFO 2016-12-21 11:37:53,620 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,620 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling mapreduce_history_server_rpc_latency with UUID ba7958f6-ef66-4d18-bec3-e5a46f08807f INFO 2016-12-21 11:37:53,620 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,621 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling mapreduce_history_server_cpu with UUID 7a92d5d1-707c-46dd-979d-51489f6d6ffb INFO 2016-12-21 11:37:53,621 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,621 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_cpu with UUID bdac8503-92ad-4fee-b7ea-372855e42595 INFO 2016-12-21 11:37:53,621 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,621 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling secondary_namenode_process with UUID 32bf8fcd-d1ae-4a48-badb-3659d581a42a INFO 2016-12-21 11:37:53,621 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,621 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_hdfs_pending_deletion_blocks with UUID 986a4c4f-754e-409e-a19c-99d3d2f555c3 INFO 2016-12-21 11:37:53,621 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,622 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_ha_health with UUID c147592b-3f5b-4dd4-a7d6-c8a3f09868f8 INFO 2016-12-21 11:37:53,622 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,622 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_heap_usage with UUID 1d407c40-89af-4af2-9d9d-bcfd49d15ad5 INFO 2016-12-21 11:37:53,622 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,622 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_health_summary with UUID 4a389e92-96c4-4ed2-9e8b-ac864829a596 INFO 2016-12-21 11:37:53,622 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,622 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_unmounted_data_dir with UUID f4b2680c-8552-4a29-99cf-1feb9cdd0afd INFO 2016-12-21 11:37:53,622 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,622 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_process with UUID ca1cf4cd-7836-4600-851c-e4c7474f0549 INFO 2016-12-21 11:37:53,623 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,623 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_hdfs_blocks_health with UUID d7d65a7a-cb94-42f8-a250-0bfd002fe288 INFO 2016-12-21 11:37:53,623 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,623 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_webui with UUID 3089e3b2-41b6-4254-818e-a167d66f2766 INFO 2016-12-21 11:37:53,623 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,623 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_webui with UUID 85cf8323-b24a-4951-8278-b26061c35cbb INFO 2016-12-21 11:37:53,623 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,623 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling datanode_storage with UUID dea9037f-b340-4bd9-9a21-3bc13815f50f INFO 2016-12-21 11:37:53,624 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,624 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_client_rpc_queue_latency_daily with UUID 20d5db5f-657a-4746-a716-13637c548774 INFO 2016-12-21 11:37:53,624 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,624 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_service_rpc_queue_latency_daily with UUID e9c63a31-3531-4b40-8562-614db7f20d60 INFO 2016-12-21 11:37:53,624 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,624 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_client_rpc_processing_latency_daily with UUID c21bf200-5fa4-46ed-b262-80161fe899f6 INFO 2016-12-21 11:37:53,624 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,624 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_service_rpc_processing_latency_hourly with UUID 62accfec-c469-4ca8-858d-ec19261e904d INFO 2016-12-21 11:37:53,624 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,625 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_increase_in_storage_capacity_usage_daily with UUID 59c1438b-c07c-4ba6-8cf9-4ab24dabb508 INFO 2016-12-21 11:37:53,625 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,625 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_client_rpc_queue_latency_hourly with UUID 6ec063c0-7eca-4010-aef9-9434bb67c47f INFO 2016-12-21 11:37:53,625 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,625 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling upgrade_finalized_state with UUID ac50c03a-5a89-4726-aa4b-2c91485c6c05 INFO 2016-12-21 11:37:53,625 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,625 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_service_rpc_processing_latency_daily with UUID 7874f321-0f1c-4627-9b4f-61a91ccc4b6f INFO 2016-12-21 11:37:53,625 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,625 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_client_rpc_processing_latency_hourly with UUID 6b23e270-9078-452b-8254-fee68398a284 INFO 2016-12-21 11:37:53,626 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,626 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_increase_in_storage_capacity_usage_weekly with UUID e4b1784c-765a-4a8e-a2a5-12179e2b9c4b INFO 2016-12-21 11:37:53,626 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,626 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_service_rpc_queue_latency_hourly with UUID d7286fdd-82f0-4873-a20e-4afe0d3d58f3 INFO 2016-12-21 11:37:53,626 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,626 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling increase_nn_heap_usage_weekly with UUID 42e4ce72-6d9e-4895-8aa1-298240322e8a INFO 2016-12-21 11:37:53,626 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,626 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_directory_status with UUID 62d94540-5b98-49cc-9edf-946883e466a7 INFO 2016-12-21 11:37:53,627 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,627 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling increase_nn_heap_usage_daily with UUID 26fbbbf4-5840-4aae-936c-e4711b133bdc INFO 2016-12-21 11:37:53,627 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,627 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_hdfs_capacity_utilization with UUID fa4da8e2-6f03-4bb6-9cab-ac3f89a09186 INFO 2016-12-21 11:37:53,627 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,627 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_rpc_latency with UUID a5655dc2-cd17-4eaa-8dc8-589e8fc3fa75 INFO 2016-12-21 11:37:53,627 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,627 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling namenode_last_checkpoint with UUID 5c9e9082-8588-41cb-a468-2c2fba51832e INFO 2016-12-21 11:37:53,628 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,628 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling storm_server_process with UUID ded0cc17-d746-48d0-a765-db62005aed8f INFO 2016-12-21 11:37:53,628 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,628 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling storm_webui with UUID ce7811e8-7dab-4434-92f6-2b67133d712e INFO 2016-12-21 11:37:53,628 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,628 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling storm_supervisor_process with UUID 33506be0-ce06-48dd-8b14-92ff6edba9ef INFO 2016-12-21 11:37:53,628 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,628 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling zeppelin_server_status with UUID 612afbb3-3848-4d29-b719-70c9a762a278 INFO 2016-12-21 11:37:53,629 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,629 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling storm_nimbus_process with UUID ab5c399f-f9e5-49af-acaa-526165967897 INFO 2016-12-21 11:37:53,629 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,629 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling storm_drpc_server with UUID be6c53e6-1155-4a2e-9cb0-17ad84fff96a INFO 2016-12-21 11:37:53,629 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,629 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_nodemanager_health with UUID 93551302-784b-4d4e-9bd8-ed5112910c4d INFO 2016-12-21 11:37:53,629 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,629 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_resourcemanager_webui with UUID 22cefbd3-038e-4d94-ac2b-24d5ee870272 INFO 2016-12-21 11:37:53,629 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,630 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_resourcemanager_cpu with UUID d5d67fa0-d138-4f3f-a215-0bb490add4e7 INFO 2016-12-21 11:37:53,630 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,630 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_nodemanager_webui with UUID 8c9d8823-59d6-4ba5-be8e-742906955689 INFO 2016-12-21 11:37:53,630 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,630 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_resourcemanager_rpc_latency with UUID 778ef7ee-3fd9-4ed6-b3d1-b45f31b2e524 INFO 2016-12-21 11:37:53,630 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,630 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling nodemanager_health_summary with UUID f2636b83-7564-4c51-8dd6-94b68a78e364 INFO 2016-12-21 11:37:53,630 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,630 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling yarn_app_timeline_server_webui with UUID 5feb6de0-7691-40c6-852d-2a03df2f7115 INFO 2016-12-21 11:37:53,631 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,631 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling zookeeper_server_process with UUID d8ac9029-b050-459f-9d81-f07af972a332 INFO 2016-12-21 11:37:53,631 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,631 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling SPARK_JOBHISTORYSERVER_PROCESS with UUID b9577f10-2a3d-413c-892e-7dd6130284b3 INFO 2016-12-21 11:37:53,631 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,631 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling hive_metastore_process with UUID 779638e2-1ed2-431a-bddb-b0e80b39bec6 INFO 2016-12-21 11:37:53,631 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,631 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling hive_server_process with UUID 4d384cab-3757-457f-9cfe-f058f22ef242 INFO 2016-12-21 11:37:53,632 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,632 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling hive_webhcat_server_status with UUID 05d13d46-2eab-49d2-8b88-94fee34fc960 INFO 2016-12-21 11:37:53,632 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,632 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ams_metrics_monitor_process with UUID 28bb5d5a-faa5-43fd-812b-e8d573968ee0 INFO 2016-12-21 11:37:53,632 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,632 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ams_metrics_collector_process with UUID 792e904a-247c-4039-8718-7fed92a42237 INFO 2016-12-21 11:37:53,632 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,632 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ams_metrics_collector_hbase_master_process with UUID a8694285-722a-433c-8b1a-37edcd045a5b INFO 2016-12-21 11:37:53,632 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,633 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ams_metrics_collector_hbase_master_cpu with UUID 4cdab24d-5dbb-49e0-b8b1-ff1f64a69fb8 INFO 2016-12-21 11:37:53,633 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,633 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ams_metrics_collector_autostart with UUID 6a8e7049-a5de-44d7-ba23-85b59c6c270e INFO 2016-12-21 11:37:53,633 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,633 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling grafana_webui with UUID 6fca9975-76f9-4618-aa57-7248dd460374 INFO 2016-12-21 11:37:53,633 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,633 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ambari_agent_disk_usage with UUID 72699571-1941-4e36-8e91-74024dfa1a18 INFO 2016-12-21 11:37:53,633 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,633 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling infra_solr with UUID 153bfb16-0960-4476-974a-20055067f373 INFO 2016-12-21 11:37:53,634 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,634 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ranger_admin_process with UUID 2bd1e1b7-42ec-46d8-a0b0-e97ecdc92abb INFO 2016-12-21 11:37:53,634 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,634 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ranger_admin_password_check with UUID 9c3950fe-e67c-47dd-b016-6cdeae90854f INFO 2016-12-21 11:37:53,634 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,634 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling ranger_usersync_process with UUID 645639e4-1e2f-409d-901d-7d33fd9db926 INFO 2016-12-21 11:37:53,634 scheduler.py:287 - Adding job tentatively -- it will be properly scheduled when the scheduler starts INFO 2016-12-21 11:37:53,634 AlertSchedulerHandler.py:356 - [AlertScheduler] Scheduling metadata_server_webui with UUID ff69b921-cc02-4862-a9b6-968375f8449a INFO 2016-12-21 11:37:53,635 AlertSchedulerHandler.py:156 - [AlertScheduler] Starting ; currently running: False INFO 2016-12-21 11:37:53,638 hostname.py:98 - Read public hostname 'sandbox.hortonworks.com' using socket.getfqdn() INFO 2016-12-21 11:37:53,643 logger.py:71 - call[['test', '-w', '/']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,648 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,648 logger.py:71 - call[['test', '-w', '/dev']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,652 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,652 logger.py:71 - call[['test', '-w', '/dev/shm']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,657 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,657 logger.py:71 - call[['test', '-w', '/run']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,661 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,662 logger.py:71 - call[['test', '-w', '/sys/fs/cgroup']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,666 logger.py:71 - call returned (1, '') INFO 2016-12-21 11:37:53,666 logger.py:71 - call[['test', '-w', '/mnt']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,670 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,671 logger.py:71 - call[['test', '-w', '/run/user/1023']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,675 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,675 logger.py:71 - call[['test', '-w', '/run/user/2606233']] {'sudo': True, 'timeout': 5} INFO 2016-12-21 11:37:53,679 logger.py:71 - call returned (0, '') INFO 2016-12-21 11:37:53,691 Facter.py:194 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources. INFO 2016-12-21 11:37:53,739 Controller.py:160 - Registering with sandbox.hortonworks.com (10.69.160.144) (agent='{"hardwareProfile": {"kernel": "Linux", "domain": "hortonworks.com", "physicalprocessorcount": 8, "kernelrelease": "3.10.0-514.2.2.el7.x86_64", "uptime_days": "5", "memorytotal": 32518860, "swapfree": "0.00 GB", "memorysize": 32518860, "osfamily": "redhat", "swapsize": "0.00 GB", "processorcount": 8, "netmask": "255.255.254.0", "timezone": "UTC", "hardwareisa": "x86_64", "memoryfree": 17678456, "operatingsystem": "centos", "kernelmajversion": "3.10", "kernelversion": "3.10.0", "macaddress": "12:28:30:A9:20:47", "operatingsystemrelease": "7.3.1611", "ipaddress": "10.69.160.144", "hostname": "sandbox", "uptime_hours": "121", "fqdn": "sandbox.hortonworks.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "49799496", "used": "34072636", "percent": "41%", "device": "/dev/xvda1", "mountpoint": "/", "type": "xfs", "size": "83872132"}, {"available": "16371800", "used": "0", "percent": "0%", "device": "devtmpfs", "mountpoint": "/dev", "type": "devtmpfs", "size": "16371800"}, {"available": "16259424", "used": "4", "percent": "1%", "device": "tmpfs", "mountpoint": "/dev/shm", "type": "tmpfs", "size": "16259428"}, {"available": "15271992", "used": "987436", "percent": "7%", "device": "tmpfs", "mountpoint": "/run", "type": "tmpfs", "size": "16259428"}, {"available": "97776180", "used": "61216", "percent": "1%", "device": "/dev/xvdf1", "mountpoint": "/mnt", "type": "ext3", "size": "103080224"}, {"available": "3251888", "used": "0", "percent": "0%", "device": "tmpfs", "mountpoint": "/run/user/1023", "type": "tmpfs", "size": "3251888"}, {"available": "3251888", "used": "0", "percent": "0%", "device": "tmpfs", "mountpoint": "/run/user/2606233", "type": "tmpfs", "size": "3251888"}], "hardwaremodel": "x86_64", "uptime_seconds": "438577", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.4.1.0", "agentEnv": {"transparentHugePage": "", "hostHealth": {"agentTimeStampAtReporting": 1482320273730, "activeJavaProcs": [{"command": "/usr/jdk64/jdk1.8.0_60/bin/java -Dproc_namenode -Xmx1024m -Dhdp.version=2.5.0.0-1245 -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.0.0-1245/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.0.0-1245 -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop-hdfs-namenode-sandbox.hortonworks.com.log -Dhadoop.home.dir=/usr/hdp/2.5.0.0-1245/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=128m -XX:MaxNewSize=128m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190936 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -XX:OnOutOfMemoryError=\\"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node\\" -Dorg.mortbay.jetty.Request.maxFormContentSize=-1 -server -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=128m -XX:MaxNewSize=128m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190936 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -XX:OnOutOfMemoryError=\\"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node\\" -Dorg.mortbay.jetty.Request.maxFormContentSize=-1 -server -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=128m -XX:MaxNewSize=128m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190936 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -XX:OnOutOfMemoryError=\\"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node\\" -Dorg.mortbay.jetty.Request.maxFormContentSize=-1 -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.NameNode", "pid": 1671, "hadoop": true, "user": "hdfs"}, {"command": "/bin/java -cp /usr/local/neo4j-community-3.0.7/plugins:/usr/local/neo4j-community-3.0.7/conf:/usr/local/neo4j-community-3.0.7/lib/*:/usr/local/neo4j-community-3.0.7/plugins/* -server -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:hashCode=5 -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFinalNonStaticFields -XX:+DisableExplicitGC -Djdk.tls.ephemeralDHKeySize=2048 -Dunsupported.dbms.udc.source=tarball -Dfile.encoding=UTF-8 org.neo4j.server.CommunityEntryPoint --home-dir=/usr/local/neo4j-community-3.0.7 --config-dir=/usr/local/neo4j-community-3.0.7/conf", "pid": 11721, "hadoop": false, "user": "root"}, {"command": "/usr/jdk64/jdk1.8.0_60/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/ambari-metrics-collector/hs_err_pid%p.log -Djava.io.tmpdir=/var/lib/ambari-metrics-collector/hbase-tmp -Djava.library.path=/usr/lib/ams-hbase/lib/hadoop-native/ -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/ambari-metrics-collector/gc.log-201612210605 -Xms1408m -Xmx1408m -Xmn192m -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Dhbase.log.dir=/var/log/ambari-metrics-collector -Dhbase.log.file=hbase-ams-master-sandbox.hortonworks.com.log -Dhbase.home.dir=/usr/lib/ams-hbase/ -Dhbase.id.str=ams -Dhbase.root.logger=INFO,RFA -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster start", "pid": 12864, "hadoop": true, "user": "ams"}, {"command": "/usr/jdk64/jdk1.8.0_60/bin/java -Xms512m -Xmx512m -Djava.library.path=/usr/lib/ams-hbase/lib/hadoop-native -XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/ambari-metrics-collector/collector-gc.log-201612210605 -cp /usr/lib/ambari-metrics-collector/*:/etc/ambari-metrics-collector/conf -Djava.net.preferIPv4Stack=true -Dams.log.dir=/var/log/ambari-metrics-collector -Dproc_timelineserver org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer", "pid": 12919, "hadoop": true, "user": "ams"}, {"command": "/usr/jdk64/jdk1.8.0_60/bin/java -Dproc_datanode -Xmx1024m -Dhdp.version=2.5.0.0-1245 -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhdp.version= -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.0.0-1245/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.0.0-1245 -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop-hdfs-datanode-sandbox.hortonworks.com.log -Dhadoop.home.dir=/usr/hdp/2.5.0.0-1245/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native:/usr/hdp/2.5.0.0-1245/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1245/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -server -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m -XX:MaxNewSize=200m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190926 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m -XX:MaxNewSize=200m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190926 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m -XX:MaxNewSize=200m -Xloggc:/var/log/hadoop/hdfs/gc.log-201612190926 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS -Dhdfs.audit.logger=INFO,DRFAAUDIT -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode", "pid": 20050, "hadoop": true, "user": "hdfs"}, {"command": "java -Dproc_rangeradmin -XX:MaxPermSize=256m -Xmx1024m -Xms1024m -Dlogdir=/var/log/ranger/admin -Dcatalina.base=/usr/hdp/2.5.0.0-1245/ranger-admin/ews -cp /usr/hdp/2.5.0.0-1245/ranger-admin/ews/webapp/WEB-INF/classes/conf:/usr/hdp/2.5.0.0-1245/ranger-admin/ews/lib/*:/usr/hdp/2.5.0.0-1245/ranger-admin/ews/ranger_jaas/*:/usr/hdp/2.5.0.0-1245/ranger-admin/ews/webapp/WEB-INF/classes/conf/ranger_jaas:/usr/jdk64/jdk1.8.0_60/lib/*:/*: org.apache.ranger.server.tomcat.EmbeddedServer", "pid": 29175, "hadoop": false, "user": "ranger"}], "liveServices": [{"status": "Unhealthy", "name": "ntpd", "desc": "\\u25cf ntpd.service - Network Time Service\\n Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)\\n Active: inactive (dead)\\n"}]}, "reverseLookup": true, "alternatives": [], "umask": "18", "firewallName": "iptables", "stackFoldersAndFiles": [{"type": "directory", "name": "/etc/hadoop"}, {"type": "directory", "name": "/etc/hive"}, {"type": "directory", "name": "/etc/oozie"}, {"type": "directory", "name": "/etc/zookeeper"}, {"type": "directory", "name": "/etc/storm"}, {"type": "directory", "name": "/etc/hive-hcatalog"}, {"type": "directory", "name": "/etc/tez"}, {"type": "directory", "name": "/etc/falcon"}, {"type": "directory", "name": "/etc/knox"}, {"type": "directory", "name": "/etc/hive-webhcat"}, {"type": "directory", "name": "/etc/kafka"}, {"type": "directory", "name": "/etc/slider"}, {"type": "directory", "name": "/etc/storm-slider-client"}, {"type": "directory", "name": "/etc/spark"}, {"type": "directory", "name": "/etc/pig"}, {"type": "directory", "name": "/etc/ranger"}, {"type": "directory", "name": "/etc/ambari-metrics-collector"}, {"type": "directory", "name": "/etc/ambari-metrics-monitor"}, {"type": "directory", "name": "/etc/atlas"}, {"type": "directory", "name": "/etc/zeppelin"}, {"type": "directory", "name": "/var/run/hadoop"}, {"type": "directory", "name": "/var/run/hive"}, {"type": "directory", "name": "/var/run/zookeeper"}, {"type": "directory", "name": "/var/run/ranger"}, {"type": "directory", "name": "/var/run/ambari-metrics-collector"}, {"type": "directory", "name": "/var/log/hadoop"}, {"type": "directory", "name": "/var/log/hive"}, {"type": "directory", "name": "/var/log/oozie"}, {"type": "directory", "name": "/var/log/zookeeper"}, {"type": "directory", "name": "/var/log/storm"}, {"type": "directory", "name": "/var/log/hive-hcatalog"}, {"type": "directory", "name": "/var/log/falcon"}, {"type": "directory", "name": "/var/log/webhcat"}, {"type": "directory", "name": "/var/log/hadoop-yarn"}, {"type": "directory", "name": "/var/log/hadoop-mapreduce"}, {"type": "directory", "name": "/var/log/knox"}, {"type": "directory", "name": "/var/log/kafka"}, {"type": "directory", "name": "/var/log/spark"}, {"type": "directory", "name": "/var/log/ranger"}, {"type": "directory", "name": "/var/log/ambari-metrics-collector"}, {"type": "directory", "name": "/var/log/ambari-metrics-monitor"}, {"type": "directory", "name": "/var/log/atlas"}, {"type": "directory", "name": "/var/log/zeppelin"}, {"type": "directory", "name": "/usr/lib/flume"}, {"type": "directory", "name": "/usr/lib/storm"}, {"type": "directory", "name": "/usr/lib/ambari-metrics-collector"}, {"type": "directory", "name": "/var/lib/hive"}, {"type": "directory", "name": "/var/lib/oozie"}, {"type": "directory", "name": "/var/lib/hadoop-hdfs"}, {"type": "directory", "name": "/var/lib/hadoop-yarn"}, {"type": "directory", "name": "/var/lib/hadoop-mapreduce"}, {"type": "directory", "name": "/var/lib/knox"}, {"type": "directory", "name": "/var/lib/slider"}, {"type": "directory", "name": "/var/lib/spark"}, {"type": "directory", "name": "/var/lib/ranger"}, {"type": "directory", "name": "/var/lib/ambari-metrics-collector"}, {"type": "directory", "name": "/var/lib/atlas"}, {"type": "directory", "name": "/var/lib/zeppelin"}, {"type": "directory", "name": "/tmp/hive"}, {"type": "directory", "name": "/tmp/ambari-qa"}, {"type": "directory", "name": "/tmp/hadoop-hdfs"}, {"type": "directory", "name": "/tmp/hadoop-yarn"}, {"type": "directory", "name": "/hadoop/zookeeper"}, {"type": "directory", "name": "/hadoop/hdfs"}, {"type": "directory", "name": "/hadoop/storm"}, {"type": "directory", "name": "/hadoop/yarn"}, {"type": "directory", "name": "/kafka-logs"}], "existingUsers": [{"status": "Available", "name": "hive", "homeDir": "/home/hive"}, {"status": "Available", "name": "storm", "homeDir": "/home/storm"}, {"status": "Available", "name": "zookeeper", "homeDir": "/home/zookeeper"}, {"status": "Available", "name": "ams", "homeDir": "/home/ams"}, {"status": "Available", "name": "tez", "homeDir": "/home/tez"}, {"status": "Available", "name": "spark", "homeDir": "/home/spark"}, {"status": "Available", "name": "ambari-qa", "homeDir": "/home/ambari-qa"}, {"status": "Available", "name": "kafka", "homeDir": "/home/kafka"}, {"status": "Available", "name": "hdfs", "homeDir": "/home/hdfs"}, {"status": "Available", "name": "yarn", "homeDir": "/home/yarn"}, {"status": "Available", "name": "mapred", "homeDir": "/home/mapred"}, {"status": "Available", "name": "knox", "homeDir": "/home/knox"}, {"status": "Available", "name": "hcat", "homeDir": "/home/hcat"}, {"status": "Available", "name": "hue", "homeDir": "/home/hue"}, {"status": "Available", "name": "ranger", "homeDir": "/home/ranger"}, {"status": "Available", "name": "atlas", "homeDir": "/home/atlas"}, {"status": "Available", "name": "zeppelin", "homeDir": "/home/zeppelin"}, {"status": "Available", "name": "oozie", "homeDir": "/home/oozie"}, {"status": "Available", "name": "falcon", "homeDir": "/var/lib/falcon"}, {"status": "Available", "name": "kms", "homeDir": "/home/kms"}], "firewallRunning": false}, "timestamp": 1482320273695, "hostname": "sandbox.hortonworks.com", "responseId": -1, "publicHostname": "sandbox.hortonworks.com"}') INFO 2016-12-21 11:37:53,740 NetUtil.py:62 - Connecting to https://sandbox.hortonworks.com:8440/connection_info INFO 2016-12-21 11:37:53,792 security.py:100 - SSL Connect being called.. connecting to the server INFO 2016-12-21 11:37:53,843 security.py:61 - SSL connection established. Two-way SSL authentication is turned off on the server. INFO 2016-12-21 11:37:55,625 Controller.py:186 - Registration Successful (response id = 0) INFO 2016-12-21 11:37:55,625 ClusterConfiguration.py:119 - Updating cached configurations for cluster Sandbox INFO 2016-12-21 11:37:55,657 RecoveryManager.py:577 - RecoverConfig = {u'components': u'METRICS_COLLECTOR', u'maxCount': u'6', u'maxLifetimeCount': u'1024', u'recoveryTimestamp': 1482320274002, u'retryGap': u'5', u'type': u'AUTO_START', u'windowInMinutes': u'60'} INFO 2016-12-21 11:37:55,657 RecoveryManager.py:677 - ==> Auto recovery is enabled with maximum 6 in 60 minutes with gap of 5 minutes between and lifetime max being 1024. Enabled components - METRICS_COLLECTOR INFO 2016-12-21 11:37:55,657 AmbariConfig.py:273 - Updating config property (agent.check.remote.mounts) with value (false) INFO 2016-12-21 11:37:55,657 AmbariConfig.py:273 - Updating config property (agent.auto.cache.update) with value (true) INFO 2016-12-21 11:37:55,657 AmbariConfig.py:273 - Updating config property (agent.check.mounts.timeout) with value (0) INFO 2016-12-21 11:37:55,660 RecoveryManager.py:204 - New status, desired status is set to STARTED for INFRA_SOLR INFO 2016-12-21 11:37:55,660 RecoveryManager.py:204 - New status, desired status is set to STARTED for INFRA_SOLR_CLIENT INFO 2016-12-21 11:37:55,660 RecoveryManager.py:204 - New status, desired status is set to STARTED for METRICS_MONITOR INFO 2016-12-21 11:37:55,660 RecoveryManager.py:204 - New status, desired status is set to STARTED for METRICS_COLLECTOR INFO 2016-12-21 11:37:55,726 RecoveryManager.py:204 - New status, desired status is set to STARTED for METRICS_GRAFANA INFO 2016-12-21 11:37:55,726 RecoveryManager.py:204 - New status, desired status is set to STARTED for ATLAS_CLIENT INFO 2016-12-21 11:37:55,726 RecoveryManager.py:204 - New status, desired status is set to STARTED for ATLAS_SERVER INFO 2016-12-21 11:37:55,726 RecoveryManager.py:204 - New status, desired status is set to STARTED for SECONDARY_NAMENODE INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for HDFS_CLIENT INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for DATANODE INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for NAMENODE INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for HIVE_SERVER INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for MYSQL_SERVER INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for HIVE_METASTORE INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for HIVE_CLIENT INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for HCAT INFO 2016-12-21 11:37:55,727 RecoveryManager.py:204 - New status, desired status is set to STARTED for WEBHCAT_SERVER INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for KAFKA_BROKER INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for KERBEROS_CLIENT INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for HISTORYSERVER INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for MAPREDUCE2_CLIENT INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for OOZIE_CLIENT INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for PIG INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for RANGER_ADMIN INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for RANGER_USERSYNC INFO 2016-12-21 11:37:55,728 RecoveryManager.py:204 - New status, desired status is set to STARTED for SLIDER INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for SPARK_CLIENT INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for SPARK_JOBHISTORYSERVER INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to INSTALLED for SUPERVISOR INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to INSTALLED for NIMBUS INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to INSTALLED for DRPC_SERVER INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to INSTALLED for STORM_UI_SERVER INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for TEZ_CLIENT INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for NODEMANAGER INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for YARN_CLIENT INFO 2016-12-21 11:37:55,729 RecoveryManager.py:204 - New status, desired status is set to STARTED for APP_TIMELINE_SERVER INFO 2016-12-21 11:37:55,730 RecoveryManager.py:204 - New status, desired status is set to STARTED for RESOURCEMANAGER INFO 2016-12-21 11:37:55,730 RecoveryManager.py:204 - New status, desired status is set to STARTED for ZEPPELIN_MASTER INFO 2016-12-21 11:37:55,730 RecoveryManager.py:204 - New status, desired status is set to STARTED for ZOOKEEPER_SERVER INFO 2016-12-21 11:37:55,730 RecoveryManager.py:204 - New status, desired status is set to INSTALLED for ZOOKEEPER_CLIENT INFO 2016-12-21 11:37:55,730 ActionQueue.py:104 - Adding STATUS_COMMAND for component INFRA_SOLR of service AMBARI_INFRA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,739 ActionQueue.py:104 - Adding STATUS_COMMAND for component INFRA_SOLR_CLIENT of service AMBARI_INFRA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,747 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_MONITOR of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,756 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_COLLECTOR of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,841 ActionQueue.py:104 - Adding STATUS_COMMAND for component METRICS_GRAFANA of service AMBARI_METRICS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,850 ActionQueue.py:104 - Adding STATUS_COMMAND for component ATLAS_CLIENT of service ATLAS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,859 ActionQueue.py:104 - Adding STATUS_COMMAND for component ATLAS_SERVER of service ATLAS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,867 ActionQueue.py:104 - Adding STATUS_COMMAND for component SECONDARY_NAMENODE of service HDFS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,876 ActionQueue.py:104 - Adding STATUS_COMMAND for component HDFS_CLIENT of service HDFS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,884 ActionQueue.py:104 - Adding STATUS_COMMAND for component DATANODE of service HDFS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,893 ActionQueue.py:104 - Adding STATUS_COMMAND for component NAMENODE of service HDFS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,902 ActionQueue.py:104 - Adding STATUS_COMMAND for component HIVE_SERVER of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,910 ActionQueue.py:104 - Adding STATUS_COMMAND for component MYSQL_SERVER of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,919 ActionQueue.py:104 - Adding STATUS_COMMAND for component HIVE_METASTORE of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,928 ActionQueue.py:104 - Adding STATUS_COMMAND for component HIVE_CLIENT of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,937 ActionQueue.py:104 - Adding STATUS_COMMAND for component HCAT of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,945 ActionQueue.py:104 - Adding STATUS_COMMAND for component WEBHCAT_SERVER of service HIVE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,954 ActionQueue.py:104 - Adding STATUS_COMMAND for component KAFKA_BROKER of service KAFKA of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,963 ActionQueue.py:104 - Adding STATUS_COMMAND for component KERBEROS_CLIENT of service KERBEROS of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,971 ActionQueue.py:104 - Adding STATUS_COMMAND for component HISTORYSERVER of service MAPREDUCE2 of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,980 ActionQueue.py:104 - Adding STATUS_COMMAND for component MAPREDUCE2_CLIENT of service MAPREDUCE2 of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,989 ActionQueue.py:104 - Adding STATUS_COMMAND for component OOZIE_CLIENT of service OOZIE of cluster Sandbox to the queue. INFO 2016-12-21 11:37:55,997 ActionQueue.py:104 - Adding STATUS_COMMAND for component PIG of service PIG of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,006 ActionQueue.py:104 - Adding STATUS_COMMAND for component RANGER_ADMIN of service RANGER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,015 ActionQueue.py:104 - Adding STATUS_COMMAND for component RANGER_USERSYNC of service RANGER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,023 ActionQueue.py:104 - Adding STATUS_COMMAND for component SLIDER of service SLIDER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,032 ActionQueue.py:104 - Adding STATUS_COMMAND for component SPARK_CLIENT of service SPARK of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,040 ActionQueue.py:104 - Adding STATUS_COMMAND for component SPARK_JOBHISTORYSERVER of service SPARK of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,049 ActionQueue.py:104 - Adding STATUS_COMMAND for component SUPERVISOR of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,058 ActionQueue.py:104 - Adding STATUS_COMMAND for component NIMBUS of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,066 ActionQueue.py:104 - Adding STATUS_COMMAND for component DRPC_SERVER of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,075 ActionQueue.py:104 - Adding STATUS_COMMAND for component STORM_UI_SERVER of service STORM of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,084 ActionQueue.py:104 - Adding STATUS_COMMAND for component TEZ_CLIENT of service TEZ of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,092 ActionQueue.py:104 - Adding STATUS_COMMAND for component NODEMANAGER of service YARN of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,101 ActionQueue.py:104 - Adding STATUS_COMMAND for component YARN_CLIENT of service YARN of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,109 ActionQueue.py:104 - Adding STATUS_COMMAND for component APP_TIMELINE_SERVER of service YARN of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,118 ActionQueue.py:104 - Adding STATUS_COMMAND for component RESOURCEMANAGER of service YARN of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,127 ActionQueue.py:104 - Adding STATUS_COMMAND for component ZEPPELIN_MASTER of service ZEPPELIN of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,135 ActionQueue.py:104 - Adding STATUS_COMMAND for component ZOOKEEPER_SERVER of service ZOOKEEPER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,144 ActionQueue.py:104 - Adding STATUS_COMMAND for component ZOOKEEPER_CLIENT of service ZOOKEEPER of cluster Sandbox to the queue. INFO 2016-12-21 11:37:56,191 AlertSchedulerHandler.py:271 - [AlertScheduler] Caching cluster Sandbox with alert hash 1c065afe076bfd3cd6cd428f3e9331a6 INFO 2016-12-21 11:37:56,213 AlertSchedulerHandler.py:338 - [AlertScheduler] The alert mapreduce_history_server_process with UUID d7fec1e4-852d-44ae-8204-5630c4e07872 is disabled and will not be scheduled INFO 2016-12-21 11:37:56,214 AlertSchedulerHandler.py:211 - [AlertScheduler] Reschedule Summary: 1 rescheduled, 0 unscheduled INFO 2016-12-21 11:37:56,215 Controller.py:463 - Registration response from sandbox.hortonworks.com was OK INFO 2016-12-21 11:37:56,215 Controller.py:468 - Resetting ActionQueue... INFO 2016-12-21 11:37:57,972 RecoveryManager.py:185 - current status is set to STARTED for METRICS_COLLECTOR