Member since
08-17-2017
5
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3552 | 09-20-2017 01:46 PM |
09-20-2017
02:17 PM
Khaja Hussain You haven't mentioned time the workload timings when they run separately. For example, Can you run Jan 2017 data processing separately and record the number of containers it requires and the time it takes (10 min) The same way repeat for Feb 2017 data processing separately(12 min). Compare the number of containers each job demands. Then set the order policy as FAIR and set Minimum user limit to 50% for the queue, and run both the jobs parallely. Now the resource allocation should be equally distributed and observe the increased run time for the jobs as the number of containers available for each job were reduced. Also please refer the article on user-limit-factor in a yarn queue, https://community.hortonworks.com/content/supportkb/49640/what-does-the-user-limit-factor-do-when-used-in-ya.html Accept the answer if this helps for your query.
... View more