Please accept the answer, if it has solved your question ... View more

What do you have following property set to "yarn.scheduler.minimum-allocation-mb". when you set map.memory = 2G, you might end up getting a 4GB container with opt 2048. ... View more

@white wartih When you run the ambari-server setup command (Without using -j /path/to/your/installed/jdk) option then ambari provides you two options to choose your JDK JDK1.8 or JDK1.7 as following # ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'enabled' SELinux mode is 'permissive' WARNING: SELinux is set to 'permissive' mode and temporarily disabled. OK to continue [y/n] (y)? Customize user account for ambari-server daemon [y/n] (n)? Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... Do you want to change Oracle JDK [y/n] (n)? y [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7 [3] Custom JDK ============================================================================== Enter choice (1): . When you choose any of those options then ambari will download the JDK from the following locations. # grep "jdk1.7.url" /etc/ambari-server/conf/ambari.properties jdk1.7.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz # grep "jdk1.8.url" /etc/ambari-server/conf/ambari.properties jdk1.8.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz . On the ambari server the desired JDK TAR ball is placed inside the following location: # ls -l /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz -rw-r--r--. 1 root root 181238643 Jun 22 16:26 /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz . And the same JDK is pushed to all the ambari agent machines in the following location. # ls -l /var/lib/ambari-agent/tmp/jdk-8u112-linux-x64.tar.gz -rwxr-xr-x. 1 root root 183212596 Jun 9 11:12 /var/lib/ambari-agent/tmp/jdk-8u112-linux-x64.tar.gz . This is how the agents gets the JDK binary from ambari server. . So if you want to choose your own JDK then as per the Current Design and Implementation of Ambari you will need to Make sure that the JDK is manually installed on all the host machine by you and on the asme path that you mentioned in the "ambari-server setup -j /path/to/your/installed/jdk" command. And this is what Ambari Server Documentation says: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.0/bk_ambari-installation/content/setup_options.html . ... View more

Hi, Today i deployed Hortonworks cluster in AWS cloud. Even i am facing same issue while starting AMS collector. Please find the log files for the same. ------------------------------------ netstat -tulpn | grep 6188 No result. So AMS is down. ----------------------------------- df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 992M 56K 992M 1% /dev tmpfs 1001M 12K 1001M 1% /dev/shm /dev/xvda1 30G 4.5G 25G 16% / ----------------------------------- ambari-metrics-collector.log 2017-11-09 11:52:33,843 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server ip-172-31-19-191.ap-south-1.compute.internal/172.31.19.191:2181. Will not attempt to authenticate using SASL (unknown error) 2017-11-09 11:52:33,843 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to ip-172-31-19-191.ap-south-1.compute.internal/172.31.19.191:2181, initiating session 2017-11-09 11:52:33,844 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect 2017-11-09 11:52:35,033 INFO org.apache.helix.manager.zk.ZkClient: Closing zkclient: State:CONNECTING sessionid:0x0 local:null remoteserver:null lastZxid:0 xid:1 sent:24 recv:0 queuedpkts:0 pendingresp:0 queuedevents:0 2017-11-09 11:52:35,033 INFO org.I0Itec.zkclient.ZkEventThread: Terminate ZkClient event thread. 2017-11-09 11:52:35,936 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server ip-172-31-21-67.ap-south-1.compute.internal/172.31.21.67:2181. Will not attempt to authenticate using SASL (unknown error) 2017-11-09 11:52:35,936 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to ip-172-31-21-67.ap-south-1.compute.internal/172.31.21.67:2181, initiating session 2017-11-09 11:52:36,039 INFO org.apache.zookeeper.ZooKeeper: Session: 0x0 closed 2017-11-09 11:52:36,039 INFO org.apache.helix.manager.zk.ZkClient: Closed zkclient 2017-11-09 11:52:36,039 ERROR org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore: org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000 2017-11-09 11:52:36,040 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore failed in state INITED; cause: org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.MetricsSystemInitializationException: Unable to initialize HA controller org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.MetricsSystemInitializationException: Unable to initialize HA controller at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.initializeSubsystem(HBaseTimelineMetricStore.java:118) at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.serviceInit(HBaseTimelineMetricStore.java:96) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:84) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:137) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:147) Caused by: org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000 at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:60) at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:69) at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:96) at org.apache.helix.manager.zk.ZKHelixAdmin.<init>(ZKHelixAdmin.java:92) at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.MetricCollectorHAController.initializeHAController(MetricCollectorHAController.java:124) at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.initializeSubsystem(HBaseTimelineMetricStore.java:115) ... 7 more 2017-11-09 11:52:36,041 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down for session: 0x0 2017-11-09 11:52:36,046 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer failed in state INITED; cause: org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.MetricsSystemInitializationException: Unable to initialize HA controller org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.MetricsSystemInitializationException: Unable to initialize HA controller at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.initializeSubsystem(HBaseTimelineMetricStore.java:118) at org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.serviceInit(HBaseTimelineMetricStore.java:96) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:84) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:137) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:147) Caused by: org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000 ------------------------------------ Security Group Added 6188 as custom TCP inbound to my security group also. ------------------------------------ Thanks in advance. Venkateswara Reddy B ... View more

@white wartih Wow!! good finding , I had written an article on the same sometime back: https://community.hortonworks.com/articles/42872/why-ambari-host-might-have-different-public-host-n.html ... View more