question Re: How can we isolate the jobs running in the same queue? in Support Questions

How can we isolate the jobs running in the same queue?

prasad4u_com — Mon, 01 Feb 2016 22:20:54 GMT

I created four node labels, assigned 4 nodes to each node label. Now, the jobs are running fine in the specified queues and on assigned nodes through node labels. Let us consider a case, two jobs are running in the same queue. Does one job which is acquiring more cluster resources will kill another job in the same queue? How can we isolate two jobs without impacting each other? Is there any solution?

Re: How can we isolate the jobs running in the same queue?

aervits — Mon, 01 Feb 2016 22:23:25 GMT

take a look at job preemption, new feature in YARN. @Ram D you can also raise and lower the priority of each job.

Command to interact with Map Reduce Jobs.

COMMAND_OPTION	Description
-submit job-file	Submits the job.
-status job-id	Prints the map and reduce completion percentage and all job counters.
-counter job-id group-name counter-name	Prints the counter value.
-kill job-id	Kills the job.
-events job-id from-event-# #-of-events	Prints the events' details received by jobtracker for the given range.
-history [all]jobOutputDir	Prints job details, failed and killed tip details. More details about the job such as successful tasks and task attempts made for each task can be viewed by specifying the [all] option.
-list [all]	Displays jobs which are yet to complete. -list all displays all jobs.
-kill-task task-id	Kills the task. Killed tasks are NOT counted against failed attempts.
-fail-task task-id	Fails the task. Failed tasks are counted against failed attempts.
-set-priority job-id priority	Changes the priority of the job. Allowed priority values are VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW

Re: How can we isolate the jobs running in the same queue?

prasad4u_com — Mon, 01 Feb 2016 22:36:37 GMT

How can we isolate two jobs without impacting each other? My requirement is jobs must be run wihtout impacting each other in the same queue. By setting same priority to both jobs is the solution ?

Re: How can we isolate the jobs running in the same queue?

aervits — Mon, 01 Feb 2016 22:38:43 GMT

@Ram D Changes the priority of the job. Allowed priority values are VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW. Change one job's priority from NORMAL and it will work. You can also look at preemption.