question Re: Introduction of HBase namespaces into a pre-existing application? in Archives of Support Questions (Read Only)

Introduction of HBase namespaces into a pre-existing application?

slm — Mon, 22 Feb 2016 01:25:26 GMT

I've inherited the support role for an application that writes all of its tables to the default namespace in HBase. I'd like to be able to leverage namespaces so that I can manage a centralized instance of HBase that many instances of our application can then be configured/reconfigured to use.

Questions

I understand how to use namespaces but was wondering if there was a easy way to facilitate this through HBase/Hadoop on the backend?
What's required to use namespaces from our application's standpoint?
Can our application simply do a "use <namespace>" and from that point be referencing a specific namespace's tables?

Re: Introduction of HBase namespaces into a pre-existing application?

aervits — Mon, 22 Feb 2016 09:39:36 GMT

@Sam Mingolelli

namespaces have been available since 0.96 and not very well documented. Here's a scenario I was able to dig up to attempt answering your question.

Step 1: Superuser (e.g. user hbase) creates namespace foo.

create_namespace ‘foo’

Step 2: Admin gives dba-bar full permissions to the namespace:

grant ’dba-bar', 'RWXCA', '@foo’

Note: namespaces are prefixed by @.

Step 3: dba-bar creates tables within the namespace:

create ’foo:t1', 'f1’

Step 4: dba-bar hands out permissions to the tables:

grant ‘user-x’, ‘RWXCA’, ‘foo:t1’

Note: All users will be able to see namespaces and tables within namespaces, but not the data.

The next best source of information would be the umbrela Jira for namespaces https://issues.apache.org/jira/browse/HBASE-8015. It has a design document in PDF form https://issues.apache.org/jira/secure/attachment/12580245/Namespace%20Design.pdf

Outside of that, please feel free to read the namespace source code or unit tests https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/TestNamespace.java. Bottom line, to use namespaces see above, to make your current application compliant with namespaces, change the code to point to the tables within namespaces, I already suggested to use Clone Snapshot and Export Snapshot tools or write Mapreduce to export/import into new table within a new namespace. I will escalate this to our docs team to have a better document as it's obvious people are starting to use this. I would've pointed you to our docs if we had that info, that's why I sent you stack overflow link as this question has not come up before. Finally, if at some point you will want to grant authorization to namespaces using Ranger, here's a thread for that https://community.hortonworks.com/questions/17764/ranger-hbase-namespace.html.

Re: Introduction of HBase namespaces into a pre-existing application?

aervits — Mon, 22 Feb 2016 09:40:43 GMT

@Enis

@Devaraj Das @Josh Elser

Re: Introduction of HBase namespaces into a pre-existing application?

slm — Mon, 22 Feb 2016 11:44:31 GMT

@Artem Ervits

Thank you, this is a much more substantial answer to what I was looking for. In reviewing this material and in my previous researching it does not look like there's any method available via the hbase-site.xml file or top level command where you can specify a default namespace wrt client calls. So our only recourse looks to be to modify our application so that it explicitly calls out <ns>.<table> instead of what it's doing now. I was hopeful there was something along the lines of a `use <ns>` type of operation that I could utilize to "pin" our application's calls to HBase tables to a specific namespace, but that doesn't seem to be the case. At any rate I appreciate your time and look forward to some more thorough docs around namespaces.

Re: Introduction of HBase namespaces into a pre-existing application?

pminovic — Mon, 22 Feb 2016 14:44:23 GMT

Hi @Sam Mingolelli, namespaces are indeed poorly documented. Here is a list of hbase shell commands you can use:

alter_namespace, create_namespace, describe_namespace, 
drop_namespace, list_namespace, list_namespace_tables

The first 4 are self-explanatory. With list_namespace you can list all available namespaces. The system namespace is now called "hbase" and tables without namespace all go to the "default" namespace. With "list_namespace_tables <namespace>" you can list tables in a given namespace. After that you can reference created tables by 'namespace:table', note that a semicolon is used as the delimiter, not a dot like in the design document.

Re: Introduction of HBase namespaces into a pre-existing application?

Enis — Tue, 23 Feb 2016 03:37:16 GMT

Yes, HBase does not have a "use namespace" equivalent. I doubt that we will ever add one, since in most of the cases, the default namespace is the special namespace "default". However, changing an application to use a table in a namespace should be pretty trivial. Make sure that you are using TableName.valueOf() methods properly.