Hi,
I have a MapReduce job which is running over more than 170 million records. This is resulting into consuming 98% of queue resource & 89% of cluster resource . Admin team is recommending that they will create new queue with limited configuration, and I should push my job into that queue.
Here are questions, I have :-
1- How can I push my mapreduce job ("hadoop jar") with minimal change to new queue ?
2- As newly created queue has limited resources, what if queue's capacity is full ? Will it result into long run or Job failure ?
3- Is there any other optimal way to prevent job from consuming all resource, we are ok if job runs little longer.
Please advise. Any help will be great.
Thanks