question Re: Build HBase from source with Ambari deploy in Archives of Support Questions (Read Only)

Build HBase from source with Ambari deploy

vasco1 — Wed, 08 Feb 2017 19:25:48 GMT

I'd like to use Ambari but be able to build HBase from source as we need a custom co-processor. Is there a documented way of doing this or do I have to turn into a full-custom install?

Re: Build HBase from source with Ambari deploy

aervits — Wed, 08 Feb 2017 19:36:33 GMT

You can deploy HBase as it ships with HDP and then deploy the jar for Co-processor to all RS with pdsh or Ansible or anything else. Editing hbase-site will be handled via Ambari configs. You don't need to rebuild HBase source.

Re: Build HBase from source with Ambari deploy

vasco1 — Wed, 08 Feb 2017 21:54:44 GMT

@Artem we do need to rebuild HBase since we need to run custom co-processors from Thrift. We want to do something like is described here: https://issues.apache.org/jira/browse/HBASE-5600 "We could create a thrift method to take the name of the class, method, and an array of params and then call coprocessorExec". If there's a better way to run a custom co-processor and access it from Thrift I'm all ears. But this brings me back to a good way to run a custom build of HBase with Ambari.

Re: Build HBase from source with Ambari deploy

smohanty — Wed, 08 Feb 2017 23:44:02 GMT

End of the day, Ambari calls yum install (or the equivalent zypper/apt) to install the hbase packages. Post install if you replace the jars or other libraries then it will be transparent to Ambari (e.g. in /usr/hdp/2.6.0.0-457/hbase/lib). Of course this means that if you do Add Host then the same steps have to repeated.

Re: Build HBase from source with Ambari deploy

vasco1 — Wed, 08 Feb 2017 23:57:02 GMT

That's what I was thinking. Was just wondering if there was a way to define a custom package somehow. Thanks!

Re: Build HBase from source with Ambari deploy

elserj — Thu, 09 Feb 2017 00:17:11 GMT

You're probably going to have a rough time trying to build a Thrift server inside of a coprocessor.

Why not just create a Thrift server external to HBase which can access your custom coprocessor via the standard protobuf RPCs?

Re: Build HBase from source with Ambari deploy

aervits — Thu, 09 Feb 2017 00:58:01 GMT

@Vasco Pinho this is an interesting thread and I'm curious which path you'll take here, please accept the answer that best addresses your question. Thanks.