Is there a way to connect to HBase using C#?

[#1 · Accepted solution]

I believe a native .net client is on the roadmap, right now you can use the REST API or two suggestions in the comments

[#2]

https://github.com/hdinsight/hbase-sdk-for-net

[#3]

Here's another one https://hbasenet.codeplex.com/

[#4]

hbasenet worked... thanks

[#5]

The HDinsight SDK seems only to work with HD Insight on specific Hbase releases... 0.98,wonder if anyone had luck working it out on HDP or with hbase version 1+

[#6]

If you use REST it doesn't matter whether it's 0.98 or 1.x. Here is non HdInsight version http://hbase.apache.org/book.html#_rest

[#7]

As has been mentioned in this thread, there is no native C# HBase client. however, there are several options for interacting with HBase from C#.

1. C# HBase Thrift client - Thrift allows for defining service endpoints and data models in a common format and using code generators to create language specific bindings. HBase provides a Thirft server and definitions. There are many examples online for creating a C# HBase Thrift Client. hbase-thrift-csharp
2. Marlin - Marlin is a C# client for interacting with Stargate (HBase REST API) that ultimately became hbase-sdk-for-net. I have not personally tested this against HBase 1.x+, but considering it uses Stargate, I expect it should work. If you are planning to use Stargate and implement your own client, which I would recommend over Thrift, make sure to use protobufs to avoid the JSON serialization overhead. Using a HTTP based approach also makes it much easier to load balance requests over multiple gateways.
3. Phoenix Query Server - Phoenix is a SQL skin on HBase. Phoenix Query Server is a REST API for submitting SQL queries to Phoenix. Here is some example code, however, I have not yet tested it. hdinsight-phoenix-sharp
4. Simba HBase ODBC Driver - Using ODBC to connect to HBase. I've heard positive feedback on this approach, especially from tools like Tableau. This is not open source and requires purchasing a license.
5. Future: Phoenix ODBC Driver - I've been told a Phoenix ODBC driver is in the works. Unfortunately, no ETA.

What we really need is an Entity or LINQ based framework, as that's how C# developers expect to interact with backend data sources. At one point, a member of the community began developing Linq2Hive, but the project appears to be no more. It may be possible to leverage Linq2Hive and the HbaseStorageHandler, but that seems like a really poor pattern. 🙂

I'm sure there are others, but hopefully this helps.

[#8]

Shane, thanks for the elaboration, I agree some further work should be done on this space... appreciate the elaboration again as this will be a great the explanation for whoever is trying to figure this out!

[#9]

Regarding

https://github.com/hdinsight/hbase-sdk-for-net

https://github.com/Azure/hdinsight-phoenix-sharp

There are two modes if you read the source code,

gateway mode is specifically targeting HDInsight cluster or other cluster requires basic auth, which the requests will go through HDInsight cluster gateway and requires basic auth anthentication to forward the request.

VNet mode is by default used in Azure Virtual Network when deploying HDInsight cluster into it with your client VMs. But it also applies to all no-HDInsight and on-prem HBase clusters, as it sends requests directly to REST server endpoints, you can try the below sample code.

var scanOptions = RequestOptions.GetDefaultOptions();
scanOptions.Port = 8090;
scanOptions.AlternativeEndpoint = "/";
var restIPs = new List<string>();
restIPs.Add("10.0.0.15");
restIPs.Add("10.0.0.16");
var client = new HBaseClient(null, options, new LoadBalancerRoundRobin(restIPs));
var scanSettings = new Scanner { batch = 10 };
ScannerInformation scannerInfo = client.CreateScanner(testTableName, scanSettings, scanOptions);
var options = RequestOptions.GetDefaultOptions();
options.Port = 8090;
options.AlternativeEndpoint = "/";
options.AlternativeHost = scannerInfo.Location.Host;
client.DeleteScanner(testTableName, scannerInfo, options);

hbase-sdk-for-net applies to all HBase versions since it uses REST APIs.

hdinsight-phoenix-sharp only targets HDI 3.4 or later or other distributions as long as its Phoenix component supports Protobuf serialization.