Support Questions
Find answers, ask questions, and share your expertise

Is there a way to connect to HBase using C#?

Solved Go to solution

Is there a way to connect to HBase using C#?

Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is there a way to connect to HBase using C#?

Mentor

I believe a native .net client is on the roadmap, right now you can use the REST API or two suggestions in the comments

View solution in original post

10 REPLIES 10

Re: Is there a way to connect to HBase using C#?

Mentor

I believe a native .net client is on the roadmap, right now you can use the REST API or two suggestions in the comments

View solution in original post

Re: Is there a way to connect to HBase using C#?

Mentor

Re: Is there a way to connect to HBase using C#?

Mentor

Here's another one https://hbasenet.codeplex.com/

Re: Is there a way to connect to HBase using C#?

Contributor

hbasenet worked... thanks

Re: Is there a way to connect to HBase using C#?

Contributor

The HDinsight SDK seems only to work with HD Insight on specific Hbase releases... 0.98,wonder if anyone had luck working it out on HDP or with hbase version 1+

Re: Is there a way to connect to HBase using C#?

Mentor

If you use REST it doesn't matter whether it's 0.98 or 1.x. Here is non HdInsight version http://hbase.apache.org/book.html#_rest

Re: Is there a way to connect to HBase using C#?

Contributor

As has been mentioned in this thread, there is no native C# HBase client. however, there are several options for interacting with HBase from C#.

  1. C# HBase Thrift client - Thrift allows for defining service endpoints and data models in a common format and using code generators to create language specific bindings. HBase provides a Thirft server and definitions. There are many examples online for creating a C# HBase Thrift Client. hbase-thrift-csharp
  2. Marlin - Marlin is a C# client for interacting with Stargate (HBase REST API) that ultimately became hbase-sdk-for-net. I have not personally tested this against HBase 1.x+, but considering it uses Stargate, I expect it should work. If you are planning to use Stargate and implement your own client, which I would recommend over Thrift, make sure to use protobufs to avoid the JSON serialization overhead. Using a HTTP based approach also makes it much easier to load balance requests over multiple gateways.
  3. Phoenix Query Server - Phoenix is a SQL skin on HBase. Phoenix Query Server is a REST API for submitting SQL queries to Phoenix. Here is some example code, however, I have not yet tested it. hdinsight-phoenix-sharp
  4. Simba HBase ODBC Driver - Using ODBC to connect to HBase. I've heard positive feedback on this approach, especially from tools like Tableau. This is not open source and requires purchasing a license.
  5. Future: Phoenix ODBC Driver - I've been told a Phoenix ODBC driver is in the works. Unfortunately, no ETA.

What we really need is an Entity or LINQ based framework, as that's how C# developers expect to interact with backend data sources. At one point, a member of the community began developing Linq2Hive, but the project appears to be no more. It may be possible to leverage Linq2Hive and the HbaseStorageHandler, but that seems like a really poor pattern. :)

I'm sure there are others, but hopefully this helps.

Re: Is there a way to connect to HBase using C#?

Contributor

Shane, thanks for the elaboration, I agree some further work should be done on this space... appreciate the elaboration again as this will be a great the explanation for whoever is trying to figure this out!

Re: Is there a way to connect to HBase using C#?

New Contributor

Regarding

https://github.com/hdinsight/hbase-sdk-for-net

https://github.com/Azure/hdinsight-phoenix-sharp

There are two modes if you read the source code,

gateway mode is specifically targeting HDInsight cluster or other cluster requires basic auth, which the requests will go through HDInsight cluster gateway and requires basic auth anthentication to forward the request.

VNet mode is by default used in Azure Virtual Network when deploying HDInsight cluster into it with your client VMs. But it also applies to all no-HDInsight and on-prem HBase clusters, as it sends requests directly to REST server endpoints, you can try the below sample code.

var scanOptions = RequestOptions.GetDefaultOptions();
scanOptions.Port = 8090;
scanOptions.AlternativeEndpoint = "/";
var restIPs = new List<string>();
restIPs.Add("10.0.0.15");
restIPs.Add("10.0.0.16");
var client = new HBaseClient(null, options, new LoadBalancerRoundRobin(restIPs));
var scanSettings = new Scanner { batch = 10 };
ScannerInformation scannerInfo = client.CreateScanner(testTableName, scanSettings, scanOptions);
var options = RequestOptions.GetDefaultOptions();
options.Port = 8090;
options.AlternativeEndpoint = "/";
options.AlternativeHost = scannerInfo.Location.Host;
client.DeleteScanner(testTableName, scannerInfo, options);

hbase-sdk-for-net applies to all HBase versions since it uses REST APIs.

hdinsight-phoenix-sharp only targets HDI 3.4 or later or other distributions as long as its Phoenix component supports Protobuf serialization.