@Jay , ok but do you have any API suggestion to perform some refresh or reload in order to create the folder , because we are working on production clusters and we not want to change something that not necessary ... View more

@Dear Jay We performed the following "procedure" per you recommendation Seems that procedure completed successfully but still stack-recommendations folder not created under /var/ambari-server where we are wrong? we change the following in ambari.properties before the procedure : pid.dir=/var/run/ambari-server bootstrap.dir=/var/run/ambari-server/bootstrap recommendations.dir=/var/run/ambari-server/stack-recommendations The procedure: NOTE - you mentioned that - "when prompted to Customize user account for ambari-server daemon?, choose y" but this question not appears during the process <br>[root@master02 ~]# ambari-server stop Using python /usr/bin/python Stopping ambari-server Waiting for server stop... Ambari Server stopped [root@master02 ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Ambari-server daemon is configured to run under user 'ambari'. Change this setting [y/n] (n)? Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... Do you want to change Oracle JDK [y/n] (n)? Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? Configuring database... Default properties detected. Using built-in database. Configuring ambari database... Checking PostgreSQL... Configuring local database... Configuring PostgreSQL... Backup for pg_hba found, reconfiguration not required Creating schema and user... done. Creating tables... done. Extracting system views... ............ Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. [root@master02 ~]# ambari-server start Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start.................................... Server started listening on 8080 DB configs consistency check: no errors and warnings were found. Ambari Server 'start' completed successfully. [root@master02 ~]# ls -ltr /var/ambari-server/bootstrap/ total 0 [root@master02 ~]# ls -ltr /var/ambari-server/ total 4 drwxr-xr-x 2 ambari root 6 Mar 12 14:10 bootstrap -rw-r--r-- 1 root root 6 Mar 12 14:10 ambari-server.pid [root@master02 ~]# find /var -name stack-recommendations [root@master02 ~]# ... View more

ambari-metrics-collector.log java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 2018-03-12 08:54:14,696 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys3423.com/43.25.76.98:61181. Will not attempt to authenticate using SASL (unknown error) 2018-03-12 08:54:14,697 WARN org.apache.zookeeper.ClientCnxn: Session 0x1621966a8020001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 2018-03-12 08:54:14,797 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=master02.sys3423.com:61181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /ams-hbase-unsecure/meta-region-server 2018-03-12 08:54:15,859 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys3423.com/43.25.76.98:61181. Will not attempt to authenticate using SASL (unknown error) 2018-03-12 08:54:15,860 WARN org.apache.zookeeper.ClientCnxn: Session 0x1621966a8020001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 2018-03-12 08:54:17,363 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys3423.com/43.25.76.98:61181. Will not attempt to authenticate using SASL (unknown error) 2018-03-12 08:54:17,364 WARN org.apache.zookeeper.ClientCnxn: Session 0x1621966a8020001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 2018-03-12 08:54:19,037 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys3423.com/43.25.76.98:61181. Will not attempt to authenticate using SASL (unknown error) 2018-03-12 08:54:19,037 WARN org.apache.zookeeper.ClientCnxn: Session 0x1621966a8020001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) ... View more

@Jay , thank you so much the "Suggested Memory settings" new values solved the problem , and now metrics collector is up , and thank you for the time that you put on this case , we are very appreciate this ... View more

@Jay you mention to "Increased heap memory for AMS collector and hbase." , please let me know what are the variable that we need to increase ? ( they from ambari GUI ) ? ... View more

@Jay we have in the cluster 3 master machines and 4 workers machine so total hosts are - 7 , so from the table we need to select the first table - (1 - 10) , an I correct ? ... View more

@jay we clean both folder and start again the metrics but we the same isshue ... View more

@Jay thank you so much , regrading the cluster size ( https://community.hortonworks.com/questions/177371/cant-start-ambari-collector.html ) , dose cluster size mean - Number of workers machines ? ... View more

the api from from this link answer about to enable all services , but what we need is only to enable / disable the auto start services , as described in the pictures ( like click on button to disable (pic1) ,and click on the button to enable (pic2) ) ... View more