Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1969 | 06-15-2020 05:23 AM | |
| 16038 | 01-30-2020 08:04 PM | |
| 2108 | 07-07-2019 09:06 PM | |
| 8245 | 01-27-2018 10:17 PM | |
| 4669 | 12-31-2017 10:12 PM |
12-04-2017
02:25 PM
on the namenode -format we get - 17/12/04 14:23:34 ERROR namenode.NameNode: Failed to start namenode.
java.io.IOException: Timed out waiting for response from loggers
... View more
12-04-2017
12:46 PM
we see from the logs also this 017-12-04 12:37:33,544 FATAL namenode.FSEditLog (JournalSet.java:mapJournalsAndReportErrors(398)) - Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [100.14.28.153:8485, 100.14.28.152:8485, 100.14.27.162:8485], stream=null))
java.io.IOException: Timed out waiting 120000ms for a quorum of nodes to respond.
... View more
12-04-2017
12:22 PM
@Geoffrey , according to my bad status , what else we can do , ( yesterday we restart all machines in the cluster , and start the services from Zk and HDFS and so on ) , so I am really stuck here
... View more
12-04-2017
12:15 PM
@Geoffrey , I already restart the HDFS but without good results after restart the status is as the current status , so I just thinking what are the next steps to resolve this problem ?
... View more
12-04-2017
11:31 AM
# host master01 master01.sys4.com has address 103.114.28.13 # host master03 master03.sys4.com has address 103.114.28.12
... View more
12-04-2017
11:28 AM
we get that: hdfs haadmin -failover -forceactive master01 master03
Illegal argument: Unable to determine service address for namenode 'master01'
... View more
12-04-2017
11:10 AM
in our ambari cluster both name node are like standby in order to force one of them to be active we do hdfs haadmin -transitionToActive --forceactive master01
Illegal argument: Unable to determine service address for namenode 'master01' but we get - Unable to determine service address what this is indicate ? and how to fix this issue ?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
12-03-2017
11:00 PM
from the out file we get that --> Getting jmx metrics from NN failed. URL: http://master03.sys56.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem
... View more
12-03-2017
07:59 PM
we have 3 masters machine in ambari cluster the first service that need to start is the zookeper server on all masters ( master01/02/03 ) but zookeeper service not start on the master01 and master02 machines from /var/log/zookeper we see the following : what chould be the problem ? 2017-12-03 19:54:29,832 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /120.14.51.19:59808
2017-12-03 19:54:29,838 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /120.14.51.19:59808
2017-12-03 19:54:29,844 - INFO [CommitProcessor:1:ZooKeeperServer@617] - Established session 0x1601dccc7e8049d with negotiated timeout 30000 for client /120.14.51.19:59808
2017-12-03 19:54:31,394 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x1601dccc7e8049d, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
2017-12-03 19:54:31,395 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for client /120.14.51.19:59808 which had sessionid 0x1601dccc7e8049d
2017-12-03 19:54:33,474 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /120.14.51.19:59810
2017-12-03 19:54:33,482 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /120.14.51.19:59810
2017-12-03 19:54:33,492 - INFO [CommitProcessor:1:ZooKeeperServer@617] - Established session 0x1601dccc7e8049e with negotiated timeout 30000 for client /120.14.51.19:59810
2017-12-03 19:54:35,150 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x1601dccc7e8049e, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
2017-12-03 19:54:35,151 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for client /120.14.51.19:59810 which had sessionid 0x1601dccc7e8049e
2017-12-03 19:54:35,378 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /120.14.51.17:50721
2017-12-03 19:54:35,388 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /120.14.51.17:50721
2017-12-03 19:54:35,397 - INFO [CommitProcessor:1:ZooKeeperServer@617] - Established session 0x1601dccc7e8049f with negotiated timeout 30000 for client /120.14.51.17:50721
^C
... View more
Labels:
- Labels:
-
Apache Ambari