Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

/usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Solved Go to solution

/usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Contributor

Hi all,

I was following this tutorial:

http://hortonworks.com/blog/using-r-and-other-non-java-languages-in-mapreduce-and-hive/

and I couldn't find hadoop-streamingxxxx.jar. I'm using a cluster with hdp-2.3.4.0-3485.

Does any know where to find it or how to add it ?

Thanks :)

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Mentor

@Zaher Mahdhi

you can find streaming jar in /usr/hdp/current/hadoop-mapreduce-client, make sure mapreduce, hdfs and yarn clients are installed on your machine.

[root@sandbox hadoop-mapreduce-client]# ls hadoop-streaming*
hadoop-streaming-2.7.1.2.3.2.0-2950.jar  hadoop-streaming.jar
[root@sandbox hadoop-mapreduce-client]# pwd
/usr/hdp/current/hadoop-mapreduce-client

View solution in original post

8 REPLIES 8
Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Mentor

@Zaher Mahdhi

you can find streaming jar in /usr/hdp/current/hadoop-mapreduce-client, make sure mapreduce, hdfs and yarn clients are installed on your machine.

[root@sandbox hadoop-mapreduce-client]# ls hadoop-streaming*
hadoop-streaming-2.7.1.2.3.2.0-2950.jar  hadoop-streaming.jar
[root@sandbox hadoop-mapreduce-client]# pwd
/usr/hdp/current/hadoop-mapreduce-client

View solution in original post

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Mentor

You can install it using Ambari, go to the host page and add mapreduce client, doesn't hurt to add yarn and hdfs client also

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Contributor

@Artem Ervits thanks :)

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

@Zaher Mahdhi

I see that you are using HDP 2.3.4.0-3485

See the following

[root@phdns01 ~]# find / -name hadoop-streaming*

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-streaming-2.7.1.2.3.4.0-3485.jar

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-streaming.jar

/usr/hdp/2.3.4.0-3485/oozie/share/lib/mapreduce-streaming/hadoop-streaming-2.7.1.2.3.4.0-3485.jar

[root@phdns01 ~]#

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Contributor
Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Contributor

When testing the example I got this error :/ :

16/03/01 01:57:29 INFO mapreduce.Job: Task Id : attempt_1456773989186_0006_r_000001_2, Status : FAILED
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
    at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
    at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
    at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:134)
    at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:244)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:459)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

@Zaher Mahdhi Open this as new question to avoid confusion

Highlighted

Re: /usr/lib/hadoop-mapreduce/hadoop-streamingxxxx.jar not found

Mentor

@Zaher Mahdhi I agree, can you please post this as a new question and provide steps to reproduce the problem.

Don't have an account?
Coming from Hortonworks? Activate your account here