Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HBase spring java connection issues

avatar
Explorer

Can someone please do a base out of the box Hortonworks sandbox VM and use one of the following repos from github or any spring framework with hbase. And tell me what the heck I am doing wrong? I have tried multiple hithub codes to see how this stuff works and I cannot get anything to connect successfully. I have been fighting this for a week now and I cannot seem to gain any ground.

Hortonworks VM - HDP™ 2.4 on Hortonworks Sandbox

Repo 1: https://github.com/spring-projects/spring-hadoop-samples/

Repo 2: https://github.com/spring-projects/spring-data-book/tree/master/hadoop

I cannot get either of these to work, I have also tried just using a basic

Configuration c = HBaseConfiguration.create();
HBaseAdmin.checkHBaseAvailable(c);

with the following hbase-site.xml

<configuration>
  
  <property>
    <name>dfs.domain.socket.path</name>
    <value>/var/lib/hadoop-hdfs/dn_socket</value>
  </property>
  
  <property>
    <name>hbase.bucketcache.ioengine</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bucketcache.percentage.in.combinedcache</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bucketcache.size</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bulkload.staging.dir</name>
    <value>/apps/hbase/staging</value>
  </property>
  
  <property>
    <name>hbase.client.keyvalue.maxsize</name>
    <value>1048576</value>
  </property>
  
  <property>
    <name>hbase.client.retries.number</name>
    <value>35</value>
  </property>
  
  <property>
    <name>hbase.client.scanner.caching</name>
    <value>100</value>
  </property>
  
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.master.classes</name>
    <value>org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.region.classes</name>
    <value>org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.regionserver.classes</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.defaults.for.version.skip</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.hregion.majorcompaction</name>
    <value>604800000</value>
  </property>
  
  <property>
    <name>hbase.hregion.majorcompaction.jitter</name>
    <value>0.50</value>
  </property>
  
  <property>
    <name>hbase.hregion.max.filesize</name>
    <value>10737418240</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.block.multiplier</name>
    <value>4</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.flush.size</name>
    <value>134217728</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.mslab.enabled</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.hstore.blockingStoreFiles</name>
    <value>10</value>
  </property>
  
  <property>
    <name>hbase.hstore.compaction.max</name>
    <value>10</value>
  </property>
  
  <property>
    <name>hbase.hstore.compactionThreshold</name>
    <value>3</value>
  </property>
  
  <property>
    <name>hbase.local.dir</name>
    <value>${hbase.tmp.dir}/local</value>
  </property>
  
  <property>
    <name>hbase.master.info.bindAddress</name>
    <value>0.0.0.0</value>
  </property>
  
  <property>
    <name>hbase.master.info.port</name>
    <value>16010</value>
  </property>
  
  <property>
    <name>hbase.master.port</name>
    <value>16000</value>
  </property>
  
  <property>
    <name>hbase.region.server.rpc.scheduler.factory.class</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.regionserver.global.memstore.size</name>
    <value>0.4</value>
  </property>
  
  <property>
    <name>hbase.regionserver.handler.count</name>
    <value>30</value>
  </property>
  
  <property>
    <name>hbase.regionserver.info.port</name>
    <value>16030</value>
  </property>
  
  <property>
    <name>hbase.regionserver.port</name>
    <value>16020</value>
  </property>
  
  <property>
    <name>hbase.regionserver.wal.codec</name>
    <value>org.apache.hadoop.hbase.regionserver.wal.WALCellCodec</value>
  </property>
  
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:8020/apps/hbase/data</value>
  </property>
  
  <property>
    <name>hbase.rpc.controllerfactory.class</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  
  <property>
    <name>hbase.rpc.protection</name>
    <value>PRIVACY</value>
  </property>
  
  <property>
    <name>hbase.rpc.timeout</name>
    <value>90000</value>
  </property>
  
  <property>
    <name>hbase.security.authentication</name>
    <value>simple</value>
  </property>
  
  <property>
    <name>hbase.security.authorization</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.superuser</name>
    <value>hbase</value>
  </property>
  
  <property>
    <name>hbase.tmp.dir</name>
    <value>/tmp/hbase-${user.name}</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>localhost</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.useMulti</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hfile.block.cache.size</name>
    <value>0.40</value>
  </property>
  
  <property>
    <name>phoenix.functions.allowUserDefinedFunctions</name>
    <value> </value>
  </property>
  
  <property>
    <name>phoenix.query.timeoutMs</name>
    <value>60000</value>
  </property>
  
  <property>
    <name>zookeeper.session.timeout</name>
    <value>60000</value>
  </property>
  
  <property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase-unsecure</value>
  </property>
  
</configuration>

I have tried asking questions about error messages and I still get no where so is it possible for someone to just try using one of those github code bases and tell me what I am doing wrong? I just cannot seem to get anywhere and it is so frustrating. I hope someone can help, thanks a lot.

1 ACCEPTED SOLUTION

avatar
Explorer

I finally got it up and running. Even tho the port we are trying to go through for hbase is 2181 there are also other ports that must be opened. So the main issue I kept running into is the ports needing to be open for 16000 and 16020, it looks like hortonworks opens 16010 and 16030 for HBase but not 16000 or 16020. Once I opened these ports I was able to connect externally via Java.

View solution in original post

21 REPLIES 21

avatar
Explorer

@Ted Yu Yeah I agree, but It was not working. I did download the hbase-site.xml and it did not work for me. I am trying it one more time with a new instance of Hortonworks to verify I did not mess anything up when I finally did try that. I will update here once I test it again.

avatar
Explorer

+1 for your help on this issue! Thanks for the help.

avatar
Master Collaborator

I used a running hbase instance (deployed with Ambari) on Openstack.

avatar
Explorer

@Ted Yu

Hmm, maybe that is the difference. I would assume that the hortonworks VM would be good to go but maybe there are some configs that are messed up.

avatar
Explorer
@Ted Yu

Was HBase running when you started the VM? Or did you have to manually start it? I have been starting it by running the start_hbase.sh on the default directory after you log in. Did you happen to do that?

avatar
Master Collaborator

Let's focus on java connection from Spring in this thread.

Please verify that you can access UI of VM - basically hbase runs in the VM.

avatar
Explorer

@Ted Yu

Yes this appears to work successfully on the VM itself without any issues by running that script mentioned above. But I still cannot develop against this because I cannot connect to it externally.

avatar
Master Collaborator

Try connecting to VM using hbase shell - with proper hbase-site.xml on classpath

avatar
Explorer

@Ted Yu

I do not have hbase shell installed on my desktop. I am not sure how to even do that. I am running windows 10. All I have done is download that spring github code and tried to connect to my local VM that is running on virtual box. Is there an easy way to install hbase shell?

avatar
Master Collaborator

I don't have access to Windows box at the moment.

Is it possible for you to find a Mac / Linux to try the Spring repo ?