question Re: Is there a way to connect to HBase using C#? in Support Questions

Is there a way to connect to HBase using C#?

nshawa — Wed, 30 Mar 2016 10:40:00 GMT

Re: Is there a way to connect to HBase using C#?

aervits — Wed, 30 Mar 2016 10:51:35 GMT

I believe a native .net client is on the roadmap, right now you can use the REST API or two suggestions in the comments

Re: Is there a way to connect to HBase using C#?

aervits — Wed, 30 Mar 2016 10:53:27 GMT

https://github.com/hdinsight/hbase-sdk-for-net

Re: Is there a way to connect to HBase using C#?

aervits — Wed, 30 Mar 2016 10:54:08 GMT

Here's another one https://hbasenet.codeplex.com/

Re: Is there a way to connect to HBase using C#?

nshawa — Wed, 30 Mar 2016 11:46:37 GMT

The HDinsight SDK seems only to work with HD Insight on specific Hbase releases... 0.98,wonder if anyone had luck working it out on HDP or with hbase version 1+

Re: Is there a way to connect to HBase using C#?

aervits — Wed, 30 Mar 2016 19:04:23 GMT

If you use REST it doesn't matter whether it's 0.98 or 1.x. Here is non HdInsight version http://hbase.apache.org/book.html#_rest

Re: Is there a way to connect to HBase using C#?

skumpf — Wed, 30 Mar 2016 19:05:20 GMT

As has been mentioned in this thread, there is no native C# HBase client. however, there are several options for interacting with HBase from C#.

C# HBase Thrift client - Thrift allows for defining service endpoints and data models in a common format and using code generators to create language specific bindings. HBase provides a Thirft server and definitions. There are many examples online for creating a C# HBase Thrift Client. hbase-thrift-csharp
Marlin - Marlin is a C# client for interacting with Stargate (HBase REST API) that ultimately became hbase-sdk-for-net. I have not personally tested this against HBase 1.x+, but considering it uses Stargate, I expect it should work. If you are planning to use Stargate and implement your own client, which I would recommend over Thrift, make sure to use protobufs to avoid the JSON serialization overhead. Using a HTTP based approach also makes it much easier to load balance requests over multiple gateways.
Phoenix Query Server - Phoenix is a SQL skin on HBase. Phoenix Query Server is a REST API for submitting SQL queries to Phoenix. Here is some example code, however, I have not yet tested it. hdinsight-phoenix-sharp
Simba HBase ODBC Driver - Using ODBC to connect to HBase. I've heard positive feedback on this approach, especially from tools like Tableau. This is not open source and requires purchasing a license.
Future: Phoenix ODBC Driver - I've been told a Phoenix ODBC driver is in the works. Unfortunately, no ETA.

What we really need is an Entity or LINQ based framework, as that's how C# developers expect to interact with backend data sources. At one point, a member of the community began developing Linq2Hive, but the project appears to be no more. It may be possible to leverage Linq2Hive and the HbaseStorageHandler, but that seems like a really poor pattern. 🙂

I'm sure there are others, but hopefully this helps.

Re: Is there a way to connect to HBase using C#?

nshawa — Thu, 31 Mar 2016 12:04:01 GMT

hbasenet worked... thanks

Re: Is there a way to connect to HBase using C#?

nshawa — Thu, 31 Mar 2016 12:06:03 GMT

Shane, thanks for the elaboration, I agree some further work should be done on this space... appreciate the elaboration again as this will be a great the explanation for whoever is trying to figure this out!

Re: Is there a way to connect to HBase using C#?

duoxu — Sat, 02 Apr 2016 05:20:48 GMT

Regarding

https://github.com/hdinsight/hbase-sdk-for-net

https://github.com/Azure/hdinsight-phoenix-sharp

There are two modes if you read the source code,

gateway mode is specifically targeting HDInsight cluster or other cluster requires basic auth, which the requests will go through HDInsight cluster gateway and requires basic auth anthentication to forward the request.

VNet mode is by default used in Azure Virtual Network when deploying HDInsight cluster into it with your client VMs. But it also applies to all no-HDInsight and on-prem HBase clusters, as it sends requests directly to REST server endpoints, you can try the below sample code.

var scanOptions = RequestOptions.GetDefaultOptions();
scanOptions.Port = 8090;
scanOptions.AlternativeEndpoint = "/";
var restIPs = new List<string>();
restIPs.Add("10.0.0.15");
restIPs.Add("10.0.0.16");
var client = new HBaseClient(null, options, new LoadBalancerRoundRobin(restIPs));
var scanSettings = new Scanner { batch = 10 };
ScannerInformation scannerInfo = client.CreateScanner(testTableName, scanSettings, scanOptions);
var options = RequestOptions.GetDefaultOptions();
options.Port = 8090;
options.AlternativeEndpoint = "/";
options.AlternativeHost = scannerInfo.Location.Host;
client.DeleteScanner(testTableName, scannerInfo, options);

hbase-sdk-for-net applies to all HBase versions since it uses REST APIs.

hdinsight-phoenix-sharp only targets HDI 3.4 or later or other distributions as long as its Phoenix component supports Protobuf serialization.

Re: Is there a way to connect to HBase using C#?

daniel11 — Thu, 25 Aug 2016 06:41:38 GMT

Related to @Duo Xu's response, Microsoft released a preview of their .NET ProtoBuf client for Phoenix (hdinsight-phoenix-sharp) on Nuget.org in June 2016. I've found it worked well against my HDP 2.4 (HBASE 1.1.2 / Phoenix 4.4) cluster, but the API was unfamiliar to me. As I learned it, I implemented an IDbConnection, IDbCommand et al wrapper API around it, named Garuda.Data. It makes access to Phoenix from .NET code feel almost like SqlConnection, SqlCommand, etc - Check it out on Nuget: https://www.nuget.org/packages/Garuda.Data/

Blog post on the topic: http://dwdii.github.io/2016/08/13/Garuda.Data-Apache-Phoenix-For-.NET-Developers.html

Garuda.Data Github Repo: https://github.com/dwdii/GarudaUtil