Support Questions
Find answers, ask questions, and share your expertise

Setting hbase.master.dns.interface in custom hbase-site causes HBase master to fail

New Contributor

I'm trying to force HBase to use a specific set of interfaces/ip's in order to segregate network traffic. I tried setting "hbase.master.dns.interface" but the master process dies with the following error:

ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2290) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2304) Caused by: java.lang.ArrayIndexOutOfBoundsException: 3 at org.apache.hadoop.net.DNS.reverseDns(DNS.java:78) at org.apache.hadoop.net.DNS.getHosts(DNS.java:241) at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:344) at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:362) at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:341) at org.apache.hadoop.hbase.regionserver.RSRpcServices.getHostname(RSRpcServices.java:893) at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:842) at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:210) at org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:532) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:532) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:364) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2285) ... 5 more

both HBase masters and the regionservers have the same interface setup, but it seems like the masters are trying to resolve the regionserver ip's based on their hostname which won't work. Is there another option for setting the master and regionserver ip's?

4 REPLIES 4

Guru

Hi,

https://issues.apache.org/jira/browse/HBASE-13481 in theory should have fixed this case. Can you please give us more details? I think a part of your hbase-site.xml would help. Did you set the corresponding hbase.regionserver.dns.nameserver parameters?

Enis

New Contributor

I tried setting hbase.regionserver.dns.interface, but I'm getting the same errors as on the masters.

ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer
        at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2636)
        at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64)
        at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2634)
        ... 5 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 3
        at org.apache.hadoop.net.DNS.reverseDns(DNS.java:78)
        at org.apache.hadoop.net.DNS.getHosts(DNS.java:241)
        at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:344)
        at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:362)
        at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:341)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.getHostname(RSRpcServices.java:893)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:848)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:632)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:532)

Below is the corresponding section of my hbase-site.xml

    <property>
      <name>hbase.master.dns.interface</name>
      <value>bond1</value>
    </property>


    <property>
      <name>hbase.master.info.bindAddress</name>
      <value>0.0.0.0</value>
    </property>


    <property>
      <name>hbase.master.info.port</name>
      <value>16010</value>
    </property>


    <property>
      <name>hbase.master.port</name>
      <value>16000</value>
    </property>


    <property>
      <name>hbase.regionserver.dns.interface</name>
      <value>bond1</value>
    </property>



Explorer

@

Paul Edler Did you ever find a solution for your problem, I have exactly the same

Explorer

@

Paul Edler Did you ever find a solution for your problem, I have exactly the same