Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ReplicationSource warning continuously appearing in HBase logs and HBase oldWALs size in increasing.

Highlighted

ReplicationSource warning continuously appearing in HBase logs and HBase oldWALs size in increasing.

Explorer

The below is the WARNING continuously appearing in the HBase logs from the past two days and the oldWALs file size increasing. We have replication enabled with another cluster and in that cluster also the same WARNINGs are occurring in HBase logs. 

 

WARN

ReplicationSource

2-xxxxxxx-as-z14p.sys.xxxxxxx.net,60020,1557953793530-xxxxxxx-as-z21p.sys.xxxxxxx.net,60020,1561135035735 Got:

java.io.EOFException: hdfs://nameservice1/hbase/oldWALs/xxxxxxx-as-z14p.sys.xxxxxxx.net%2C60020%2C1557953793530.null0.1565404930053 not a SequenceFile

at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1852)

at org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1811)

at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1760)

at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1774)

at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:70)

at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)

at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)

at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)

at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:302)

at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:267)

at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:255)

at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:406)

at org.apache.hadoop.hbase.replication.regionserver.ReplicationWALReaderManager.openReader(ReplicationWALReaderManager.java:69)

at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceWorkerThread.openReader(ReplicationSource.java:745)

at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceWorkerThread.run(ReplicationSource.java:541)

 

I checked the oldWALs files it contain the o bytes files with date stamp from two months back and the other files with today's date stamp has data.

When it come to WALs, some of the files with today's date stamp has 0 bytes and also contains 0 bytes files from previous years.

This type of file pattern is present in both the clusters.

 

 

Don't have an account?
Coming from Hortonworks? Activate your account here