question Re: Connecting to Hadoop/Hive from .NET in Archives of Support Questions (Read Only)

Connecting to Hadoop/Hive from .NET

ssen — Sun, 27 Sep 2015 00:18:15 GMT

I am working on a solution where I will have a Hadoop cluster with Hive running and I want to send jobs and hive queries from a .NET application to be processed and get notified when they are done. What is the recommended API, library here?

Re: Connecting to Hadoop/Hive from .NET

ssen — Sun, 27 Sep 2015 00:24:13 GMT

After a bit of research, I found the hadoopsdk on codeplex is a good place to start.

As far as very basic connection examples go, try this blog for an example, but note that the connection for HDInsight is slightly different now it's all using the templeton interface, so this will get you going:

var db =newHiveConnection(
        webHCatUri:newUri("http://localhost:50111"),
        userName:(string)"hadoop", password:(string)null);var result = db.ExecuteHiveQuery("select * from w3c");

If you are looking to do full on MapReduce on HDInsight, then you probably want to take a look at the C# MapReduce examples with the sdk on codeplex.

Note that the default HDInsight install also comes with some good samples, which include a bit of data to play with and some powershell scripts and .NET code to get you started.

If there are other recommendations I am all ears.