Member since
08-10-2015
10
Posts
1
Kudos Received
0
Solutions
01-19-2016
07:32 AM
abc.jar is a 3rd party jar and I do not want to unjar and pick the mapper/reducer(there are many such files). My question here is, when I run the mapreduce program in command-line it runs with 100 mappers. The same command if I put it in shell script and run in oozie, it runs with one mapper. Is there a way that I can change this behavior? Thanks
... View more
01-18-2016
09:52 PM
@Artem Ervits Thank you. When I run the map-reduce program in command line it runs with required no of mappers, but that is not the case with running in a shell script. I will try running the same in java-action and let u know.
... View more
01-18-2016
09:13 PM
1 Kudo
Hi, I have a map-reduce program which can be called in the following manner: $ hadoop jar abc.jar DriverProg ip op I need the above mapreduce progarm to call from Oozie and it looks like I can not call DriverProg directly, instead I have to explicitly mention mapper and reducer classes. This seem to be a limitation from Oozie. Is there a way that I can call run this mapreduce program directly with Driver class ??? Since, I did not find any option to run this, I had to wrap this map-reduce program in a shell script and execute the same using shell-action in oozie. Here is the shell-script: #!/bin/bash
hadoop jar abc.jar DriverProg ip op
The above shell script runs perfectly in Oozie without any issues. However, it is always running with one mapper. The performance is totally down as I can not afford to run this Map-Reduce with single mapper. Is there anyway that I can run this shell script with required number of mappers in Oozie ??? Please assist !!! Thanks
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Oozie
12-22-2015
12:13 AM
Thank you !!! It did work .
... View more
12-21-2015
08:17 PM
Labels:
- Labels:
-
Apache Sqoop