Member since
03-31-2017
8
Posts
1
Kudos Received
0
Solutions
05-04-2017
10:59 AM
I have created HDInsight cluster on azure. I want to export hive table to SQL Server in Azure. For this I have created job in .Net SDK.
The code is as follows: I am not able to load data to the SQL Server table using this code.
... View more
- Tags:
- azure
- Data Processing
- sql
- Sqoop
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
05-03-2017
05:54 AM
I am using azure HDInsight. For Exporting Hive table to Azure SQL Server, I am submitting Sqoop job using .Net SDK. Sometime Hive table exported successfully and sometimes its having issue of IP address is different between my local PC and Azure SQL Server. Is there any way to make IP address same? Because when it is same Sqoop Job run successfully. I attached code here. Is there any credential missed in Code? Please give Solution.
Thank You
... View more
- Tags:
- .NET
- azure
- Cloud & Operations
- Sqoop
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
05-02-2017
07:24 AM
For submit Hive, Pig job in Azure HDInsight, I am using .Net SDK. After successfully output of Pig Job ,I used Pig output file to load into Hive table. Hive job executed successfully but Pig output is in BAG format , so its store data in Hive table like "(" , ")" characters. Means First and Last column of Hive table having these characters. I want to remove this characters from Hive table. Should I have to change the Pig output format or Is this anyway to remove above character from Hive Table ? Please give solution. Thank You.
... View more
- Tags:
- .NET
- Data Processing
- hdinsight
- Hive
- Pig
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
04-26-2017
10:19 AM
Thanks for reply.
It will really help me.
By mistake I wrote MapReduce Job, I should have to use HiveJob, PigJob, SqoopJob.
Thanks again.
I just go through Oozie. I didn't find exact Link for Oozie.
If I write a Pig script and than want to transfer those data in Hive. Then using Sqoop, export this data to SQL Server. How to connect all these process using Oozie.
Can you provide some reference Link?
... View more
04-26-2017
07:39 AM
1 Kudo
I have data in HDFS(Azure HDInsight) in csv format. I am using Pig to process this Data. After processing the Summarise data will be stored in Hive. And then Hive table is exported in RDBMS using Sqoop. Now I need to automate all this process. Is this possible that I will write particular method for all these 3 task in MapReduce, then run this MapReduce job, and all these task execute one by one. For create MapReduce job , I want to use .Net SDK. So my question is this possible, and if YES than suggest some steps and reference link for this Question.
Thank You.
... View more
- Tags:
- .NET
- hadoop
- Hadoop Core
- Hive
- Mapreduce
- Pig
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
04-18-2017
01:20 PM
Thanks for Reply. I was thinking like this, but confused. Now its clear.
Thanks again.
... View more
04-17-2017
02:17 PM
In Capacity Scheduler, How Resource Manage decide which Job runs on which Queue?
... View more
Labels:
04-11-2017
11:28 AM
I wrote MapReduce Job in C# using Visual Studio. Project Build Successfully but while executing I got error of connection. Like Following could
not load file or assembly ‘Microsoft.WindowsAzure.Storage,Verion=2.0.0.0,Culture=neutral,PublicKeyToken=31bf3856ad364e35’
or one of its dependencies.The system cannot find the file specified.You have
lots Error
... View more
Labels: