Created on 06-17-2014 04:19 PM - edited 09-16-2022 02:00 AM
I have a serious problem. I used Parcels and upgraded (distributed) CDH from 4.6 to 4.7. However, now HBase RegionServers will not start. I receive the following error, any help will be appreciated:
Region server nodehdp05,60020,1403044475111 reported a fatal error:
ABORTING region server nodehdp05,60020,1403044475111: Unhandled exception: Exception in createWriter
Cause:
java.io.IOException: Exception in createWriter
at org.apache.hadoop.hbase.regionserver.wal.HLogFileSystem.createWriter(HLogFileSystem.java:66)
at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriterInstance(HLog.java:709)
at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:629)
at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:573)
at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:454)
at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:378)
at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1446)
at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1417)
at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1100)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: cannot get log writer
at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:802)
at org.apache.hadoop.hbase.regionserver.wal.HLogFileSystem.createWriter(HLogFileSystem.java:60)
... 10 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec not found
Created 06-18-2014 09:01 AM
It seems that during the upgrade from CDH 4.6 to 4.7 the phoenix jar file was deleted. I should have checked when I saw this in the exception: Class org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec not found. I replaced the phoenix jar file and added back the property to the hbase-site.xml in CM and restarted HBase - it works!
Created 06-17-2014 04:41 PM
This problem has been resolved:
We were using a version of Phoenix with the following configuration in the RegionServer Advanced Configuration Snippet (Safety Valve) for hbase-site.xml:
<property>
<name>hbase.regionserver.wal.codec</name>
<value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
</property>
The HBase ResionServers were not able to start with this property. For now I’ve taken the property out of the hbase-site.xml and we’ll have to find another way to support Phoenix.
Created 06-18-2014 09:01 AM
It seems that during the upgrade from CDH 4.6 to 4.7 the phoenix jar file was deleted. I should have checked when I saw this in the exception: Class org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec not found. I replaced the phoenix jar file and added back the property to the hbase-site.xml in CM and restarted HBase - it works!