Support Questions

Find answers, ask questions, and share your expertise

Load balancing in oozie

avatar
Explorer

Hello,

 

Wanted to know if Oozie will do load balancing internally.

Requirement is we have around 200-500 shell script and which will run from different node in the cluster but we dont know how it will schedule internally. Will it do load balancing automatic if one node will have many loads on that perticular machin.

 

Please advice.

1 ACCEPTED SOLUTION

avatar
Mentor
Oozie actions (such as shell, java, etc.) are launched as regular
(1-map) MR jobs, so YARN (or JobTracker, if MR1) usually takes care of
resource management for Oozie workflows in the cluster.

View solution in original post

2 REPLIES 2

avatar
Mentor
Oozie actions (such as shell, java, etc.) are launched as regular
(1-map) MR jobs, so YARN (or JobTracker, if MR1) usually takes care of
resource management for Oozie workflows in the cluster.

avatar
Explorer

Harsh,

 

Sorry to asking you again on this. I am really in a odd situation. lets say I have 100 scripts which will fire from one node via oozie. Then how it will re-direct to other node (load balancing), I have heard about fair scheduler but need your help to undertsand it completely. Will fair scheduler will send the data across all nodes. 

 

Please help