Hi mates, I'm new to Ambari, testing it for my company as, maybe, our future Hadoop stack management an monitoring plateforme. All was working ok (tested several Zeppelin tuotrials) on single node but the mess the comes when installing on 2 vagrant-driven Vms ("master1" and "agent1" : ping-tested them together, they can communicate). Plz note "Master1" VM has 24gb of ram and 4 vcpus and agent vm has 16gb of ram and 1 vcpu. Both vm are running on the same linux workstation. Master1 has a client and server installed, agent1 has the client installed. Installed HDFS 2.7.3, Yarn 2.7.3, MapReduce2 2.7.3, Zookeeper 3.4.6 , Ambari metrics 0.1.0 on this "mini" cluster The Yarn ressource manager, perfectly working single node, don't anymore. Crawled the web but at that time didn't find how to figure the problem out. Ca anyone help ? Regards. Details below : Installation : b.1 => Got several warnings on installation : On Master1 : App Timeline Server Start, History Server Start, ResourceManager Start, SNameNode Start, Metrics Collector Start, NodeManager Start, Grafana Start
On Agent1 : Check ZooKeeper, Check HDFS, NodeManager Start, Check Ambari Metrics, Check YARN, Check MapReduce2 Log files : First of all, the logs files path for warnings informations given in the "Install, Start and Test" section don't exist :
Master : /var/lib/ambari-agent/data/output-*.txt => * ranging from 37 to 43 but the last file is 36
Agent :/var/lib/ambari-agent/data/output-*.txt => * ranging from 4 to 48 but the last file is 36 Run : HDFS, MapReduce2, Zookeeper and Ambari Metrics are said to be running fine by Ambari, but not Yarn RessouceManager wich is said to have stopped after several minutes, but in fact the logs are like this since it started :
2016-10-03 14:52:56,572 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server master1.localdomain/192.168.0.50:2181. Will not attempt to authenticate using SASL (unknown error)
2016-10-03 14:52:56,573 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(864)) - Socket connection established to master1.localdomain/192.168.0.50:2181, initiating session
2016-10-03 14:52:56,574 INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1142)) - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2016-10-03 14:52:56,675 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1227)) - Exception while executing a ZK operation.
2016-10-03 14:51:16,158 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1230)) - Retrying operation on ZK. Retry no. 695
2016-10-03 14:51:16,584 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server master1.localdomain/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-10-03 14:51:16,584 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(864)) - Socket connection established to master1.localdomain/127.0.0.1:2181, initiating session
2016-10-03 14:51:16,585 INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1142)) - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2016-10-03 14:51:16,910 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server agent1.localdomain/192.168.0.51:2181. Will not attempt to authenticate using SASL (unknown error)
2016-10-03 14:51:16,911 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(864)) - Socket connection established to agent1.localdomain/192.168.0.51:2181, initiating session
2016-10-03 14:51:16,912 INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1142)) - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[root@master1 yarn]# Logs on the Zookeeper side : 2016-10-03 14:51:16,056 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.0.50:34266
2016-10-03 14:51:16,056 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
2016-10-03 14:51:16,056 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /192.168.0.50:34266 (no session established for client)
2016-10-03 14:51:16,584 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:39082
2016-10-03 14:51:16,585 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
2016-10-03 14:51:16,585 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /127.0.0.1:39082 (no session established for client)
... View more
