I was trying using HadoopStreaming to .net core mapper & reducer HDInsight Linux cluster(HBase 3.4) I have install dotnet core on the headnode , region node and zookeeper. The dotnet core mapper is very simple public static void Main(string[] args)
{
if (args.Length > 0)
Console.SetIn(new StreamReader(new FileStream(args[0], FileMode.Open, FileAccess.Read))); string line; while ((line = Console.ReadLine()) != null) { foreach (var word in line.Split(' ')) if(word!=null &&word.Trim().Length>0) Console.WriteLine($"{word.Trim()}\t1"); } } After submitting job I got this 16/05/19 13:42:48 INFO mapreduce.Job: The url to track the job: http://hn1-xiaolv.plsmczirsz1e5czhe5z2p5tubd.hx.internal.cloudapp.net:8088/proxy/application_1463581541133_0019/ 16/05/19 13:42:48 INFO mapreduce.Job: Running job: job_1463581541133_0019
16/05/19 13:42:55 INFO mapreduce.Job: Job job_1463581541133_0019 running in uber mode : false 16/05/19 13:42:55 INFO mapreduce.Job: map 0% reduce 0% 16/05/19 13:43:00 INFO mapreduce.Job: Task Id : attempt_1463581541133_0019_m_000001_0, Status : FAILED
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 131
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
... View more
