question Re: HBase spring java connection issues in Archives of Support Questions (Read Only)

HBase spring java connection issues

jsmith — Tue, 14 Jun 2016 21:29:46 GMT

Can someone please do a base out of the box Hortonworks sandbox VM and use one of the following repos from github or any spring framework with hbase. And tell me what the heck I am doing wrong? I have tried multiple hithub codes to see how this stuff works and I cannot get anything to connect successfully. I have been fighting this for a week now and I cannot seem to gain any ground.

Hortonworks VM - HDP™ 2.4 on Hortonworks Sandbox

Repo 1: https://github.com/spring-projects/spring-hadoop-samples/

Repo 2: https://github.com/spring-projects/spring-data-book/tree/master/hadoop

I cannot get either of these to work, I have also tried just using a basic

Configuration c = HBaseConfiguration.create();
HBaseAdmin.checkHBaseAvailable(c);

with the following hbase-site.xml

<configuration>
  
  <property>
    <name>dfs.domain.socket.path</name>
    <value>/var/lib/hadoop-hdfs/dn_socket</value>
  </property>
  
  <property>
    <name>hbase.bucketcache.ioengine</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bucketcache.percentage.in.combinedcache</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bucketcache.size</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.bulkload.staging.dir</name>
    <value>/apps/hbase/staging</value>
  </property>
  
  <property>
    <name>hbase.client.keyvalue.maxsize</name>
    <value>1048576</value>
  </property>
  
  <property>
    <name>hbase.client.retries.number</name>
    <value>35</value>
  </property>
  
  <property>
    <name>hbase.client.scanner.caching</name>
    <value>100</value>
  </property>
  
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.master.classes</name>
    <value>org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.region.classes</name>
    <value>org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor</value>
  </property>
  
  <property>
    <name>hbase.coprocessor.regionserver.classes</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.defaults.for.version.skip</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.hregion.majorcompaction</name>
    <value>604800000</value>
  </property>
  
  <property>
    <name>hbase.hregion.majorcompaction.jitter</name>
    <value>0.50</value>
  </property>
  
  <property>
    <name>hbase.hregion.max.filesize</name>
    <value>10737418240</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.block.multiplier</name>
    <value>4</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.flush.size</name>
    <value>134217728</value>
  </property>
  
  <property>
    <name>hbase.hregion.memstore.mslab.enabled</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.hstore.blockingStoreFiles</name>
    <value>10</value>
  </property>
  
  <property>
    <name>hbase.hstore.compaction.max</name>
    <value>10</value>
  </property>
  
  <property>
    <name>hbase.hstore.compactionThreshold</name>
    <value>3</value>
  </property>
  
  <property>
    <name>hbase.local.dir</name>
    <value>${hbase.tmp.dir}/local</value>
  </property>
  
  <property>
    <name>hbase.master.info.bindAddress</name>
    <value>0.0.0.0</value>
  </property>
  
  <property>
    <name>hbase.master.info.port</name>
    <value>16010</value>
  </property>
  
  <property>
    <name>hbase.master.port</name>
    <value>16000</value>
  </property>
  
  <property>
    <name>hbase.region.server.rpc.scheduler.factory.class</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.regionserver.global.memstore.size</name>
    <value>0.4</value>
  </property>
  
  <property>
    <name>hbase.regionserver.handler.count</name>
    <value>30</value>
  </property>
  
  <property>
    <name>hbase.regionserver.info.port</name>
    <value>16030</value>
  </property>
  
  <property>
    <name>hbase.regionserver.port</name>
    <value>16020</value>
  </property>
  
  <property>
    <name>hbase.regionserver.wal.codec</name>
    <value>org.apache.hadoop.hbase.regionserver.wal.WALCellCodec</value>
  </property>
  
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:8020/apps/hbase/data</value>
  </property>
  
  <property>
    <name>hbase.rpc.controllerfactory.class</name>
    <value></value>
  </property>
  
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  
  <property>
    <name>hbase.rpc.protection</name>
    <value>PRIVACY</value>
  </property>
  
  <property>
    <name>hbase.rpc.timeout</name>
    <value>90000</value>
  </property>
  
  <property>
    <name>hbase.security.authentication</name>
    <value>simple</value>
  </property>
  
  <property>
    <name>hbase.security.authorization</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hbase.superuser</name>
    <value>hbase</value>
  </property>
  
  <property>
    <name>hbase.tmp.dir</name>
    <value>/tmp/hbase-${user.name}</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>localhost</value>
  </property>
  
  <property>
    <name>hbase.zookeeper.useMulti</name>
    <value>true</value>
  </property>
  
  <property>
    <name>hfile.block.cache.size</name>
    <value>0.40</value>
  </property>
  
  <property>
    <name>phoenix.functions.allowUserDefinedFunctions</name>
    <value> </value>
  </property>
  
  <property>
    <name>phoenix.query.timeoutMs</name>
    <value>60000</value>
  </property>
  
  <property>
    <name>zookeeper.session.timeout</name>
    <value>60000</value>
  </property>
  
  <property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase-unsecure</value>
  </property>
  
</configuration>

I have tried asking questions about error messages and I still get no where so is it possible for someone to just try using one of those github code bases and tell me what I am doing wrong? I just cannot seem to get anywhere and it is so frustrating. I hope someone can help, thanks a lot.

Re: HBase spring java connection issues

tyu — Tue, 14 Jun 2016 21:49:59 GMT

Last commit to https://github.com/spring-projects/spring-hadoop-samples was 11 months ago.

The other one was even older.

See if you can find newer repo.

If I have time, I plan to try the first repo.

Re: HBase spring java connection issues

jsmith — Tue, 14 Jun 2016 22:02:38 GMT

@Ted Yu I will look to see if I can find any earlier, I tried looking before but everything I found wouldn't work. But it would be a great help if you did try! I would appreciate that so much! Thanks for the help.

Re: HBase spring java connection issues

tyu — Tue, 14 Jun 2016 23:40:59 GMT

This is what I did:

cloned spring-hadoop-samples

cd spring-hadoop-samples/hbase

mvn clean package

copied hbase-site.xml from cluster to target/appassembler/etc/hbase-site.xml

sh ./target/appassembler/bin/usercount

Please try the above

Re: HBase spring java connection issues

tyu — Tue, 14 Jun 2016 23:53:21 GMT

I verified that users table was created after running the usercount example:

'users', {NAME => 'cfInfo'}

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 00:06:28 GMT

Note, here is snippet w.r.t. CLASSPATH in the sample script:

CLASSPATH=..."$REPO"/hbase-client-0.98.5-hadoop2.jar:"$REPO"/hbase-common-0.98.5-hadoop2.jar:"$REPO"/hbase-protocol-0.98.5-hadoop2.jar

You can plug in the corresponding version of hbase jars accordingly

0.98 is compatible with 1.x release - that was why the sample worked.

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 00:11:42 GMT

That was due to the following config having wrong value - plug ing actual quorum:

<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 00:12:23 GMT

@Ted Yu, is this from external source or on the same machine? Because when I run commands on the VM everything is fine. Its just connecting to the zookeeper externally is where I am having the issues. I am running my VM on the desktop but then I am using code on the desktop to connect to the VM hbase.

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 00:47:35 GMT

@Ted Yu Does this mean you did try it locally instead of on the VM?

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 00:47:49 GMT

Running sample outside VM should work as long as correct quorum is in hbase-site.xml

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 00:53:09 GMT

@Ted Yu Yeah I agree, but It was not working. I did download the hbase-site.xml and it did not work for me. I am trying it one more time with a new instance of Hortonworks to verify I did not mess anything up when I finally did try that. I will update here once I test it again.

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 00:56:27 GMT

I used a running hbase instance (deployed with Ambari) on Openstack.

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 00:58:23 GMT

@Ted Yu

Was HBase running when you started the VM? Or did you have to manually start it? I have been starting it by running the start_hbase.sh on the default directory after you log in. Did you happen to do that?

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 01:02:59 GMT

Let's focus on java connection from Spring in this thread.

Please verify that you can access UI of VM - basically hbase runs in the VM.

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 01:03:09 GMT

@Ted Yu

Hmm, maybe that is the difference. I would assume that the hortonworks VM would be good to go but maybe there are some configs that are messed up.

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 02:06:54 GMT

@Ted Yu

Yes this appears to work successfully on the VM itself without any issues by running that script mentioned above. But I still cannot develop against this because I cannot connect to it externally.

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 02:28:29 GMT

Try connecting to VM using hbase shell - with proper hbase-site.xml on classpath

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 02:57:44 GMT

@Ted Yu

I do not have hbase shell installed on my desktop. I am not sure how to even do that. I am running windows 10. All I have done is download that spring github code and tried to connect to my local VM that is running on virtual box. Is there an easy way to install hbase shell?

Re: HBase spring java connection issues

tyu — Wed, 15 Jun 2016 04:45:47 GMT

I don't have access to Windows box at the moment.

Is it possible for you to find a Mac / Linux to try the Spring repo ?

Re: HBase spring java connection issues

jsmith — Wed, 15 Jun 2016 21:53:42 GMT

I will try this some time today and see what happens. Ill update you once I have tried it.