About fernando_lopez

fernando_lopez · ‎01-11-2023

Hello everyone. Since CentOS 8 has been discontinued more than a year ago, and Rocky Linux / Alma Linux have been left occupying the same role as free RHEL's mirror distributions, I would like to know if Cloudera has already a date scheduled in the near future to start supporting any of these distributions as a base operating system for CDP base and related products. Thanks in advance

fernando_lopez · ‎01-14-2019

yarn.scheduler.capacity.maximum-am-resource-percent=0.2 yarn.scheduler.capacity.maximum-applications=10000 yarn.scheduler.capacity.node-locality-delay=40 yarn.scheduler.capacity.root.accessible-node-labels=* yarn.scheduler.capacity.root.acl_administer_queue=* yarn.scheduler.capacity.root.capacity=100 yarn.scheduler.capacity.root.default.acl_submit_applications=* yarn.scheduler.capacity.root.default.capacity=10 yarn.scheduler.capacity.root.default.maximum-capacity=30 yarn.scheduler.capacity.root.default.state=RUNNING yarn.scheduler.capacity.root.default.user-limit-factor=2 yarn.scheduler.capacity.root.queues=Hive,Zeppelin,default yarn.scheduler.capacity.queue-mappings=u:zeppelin:Zeppelin,u:hdfs:Hive,g:dl-analytics-group:Zeppelin yarn.scheduler.capacity.queue-mappings-override.enable=false yarn.scheduler.capacity.root.Hive.acl_administer_queue=* yarn.scheduler.capacity.root.Hive.acl_submit_applications=* yarn.scheduler.capacity.root.Hive.capacity=50 yarn.scheduler.capacity.root.Hive.maximum-capacity=90 yarn.scheduler.capacity.root.Hive.minimum-user-limit-percent=25 yarn.scheduler.capacity.root.Hive.ordering-policy=fair yarn.scheduler.capacity.root.Hive.ordering-policy.fair.enable-size-based-weight=false yarn.scheduler.capacity.root.Hive.priority=10 yarn.scheduler.capacity.root.Hive.state=RUNNING yarn.scheduler.capacity.root.Hive.user-limit-factor=2 yarn.scheduler.capacity.root.Zeppelin.acl_administer_queue=* yarn.scheduler.capacity.root.Zeppelin.acl_submit_applications=* yarn.scheduler.capacity.root.Zeppelin.capacity=40 yarn.scheduler.capacity.root.Zeppelin.maximum-capacity=80 yarn.scheduler.capacity.root.Zeppelin.minimum-user-limit-percent=20 yarn.scheduler.capacity.root.Zeppelin.ordering-policy=fair yarn.scheduler.capacity.root.Zeppelin.ordering-policy.fair.enable-size-based-weight=false yarn.scheduler.capacity.root.Zeppelin.priority=5 yarn.scheduler.capacity.root.Zeppelin.state=RUNNING yarn.scheduler.capacity.root.Zeppelin.user-limit-factor=3 yarn.scheduler.capacity.root.default.minimum-user-limit-percent=25 yarn.scheduler.capacity.root.default.ordering-policy=fair yarn.scheduler.capacity.root.default.ordering-policy.fair.enable-size-based-weight=false yarn.scheduler.capacity.root.default.priority=0 yarn.scheduler.capacity.root.maximum-capacity=100 yarn.scheduler.capacity.root.ordering-policy=priority-utilization yarn.scheduler.capacity.root.priority=0

fernando_lopez · ‎01-11-2019

We have defined several YARN queues. Say that you have queue Q1, where users A and B run Spark processes. If A submits a job that demands all of the queue resources, they are allocated by YARN. Subsequently, when B submits his job, he is affected by resource scarcity. We need to prevent this situation, by assigning resources more evenly between A and B (and all other incoming users), within Q1. We have already set Scheduler to Fair. Can this eager resource allocation behaviour be prevented?

Online	Offline
Last Visited	‎06-02-2023 07:43 PM

Member Since	‎09-01-2016 06:04 PM
Last Visited	‎06-02-2023 07:43 PM
Posts	52
Kudos received	13

Cloudera Community

When is CDP going to support Rocky Linux?

Re: Resource quotas within a single YARN queue

Resource quotas within a single YARN queue