About ilia987

ilia987 · ‎02-15-2019

the server and the agents are located on container(LXC) basd on proxmox virtual env 5.3 with the latest version i have some issues with ambari metrics tail /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out File "/usr/lib/python2.6/site-packages/resource_monitoring/core/metric_collector.py", line 45, in process_event self.process_host_collection_event(event) File "/usr/lib/python2.6/site-packages/resource_monitoring/core/metric_collector.py", line 80, in process_host_collection_event metrics.update(self.host_info.get_combined_disk_io_counters()) File "/usr/lib/python2.6/site-packages/resource_monitoring/core/host_info.py", line 276, in get_combined_disk_io_counters rawdict = psutil.disk_io_counters(True) File "/usr/lib/python2.6/site-packages/resource_monitoring/psutil/build/lib.linux-x86_64-2.7/psutil/__init__.py", line 1726, in disk_io_counters raise RuntimeError("couldn't find any physical disk") RuntimeError: couldn't find any physical disk

ilia987 · ‎12-27-2018

I know, i just wanted to prevent managaing multuiple configurations

ilia987 · ‎12-26-2018

i know to to configure it, for example now i have 3 types of hardares: 3 computers with 12 core cpu and 64gb ram 6 computers with 16 core and 64gb ram and 4 computers with 48 core and 256gb ram and i dont want to manage 3 configuraion files , i want to manage one, for instance each computer will allocate all cpus and 90% of ram for yarn tasks

ilia987 · ‎12-26-2018

ok, then configure a 90% mark, the main issue that i want to manage single configuraion file for all hardware

ilia987 · ‎12-26-2018

i looking for an option to set 100% and not to set fixd ram or cpu

ilia987 · ‎12-26-2018

i want to configure it via the ambari configuraion manager, so theat each host will use 100% ram and CPU

ilia987 · ‎12-25-2018

we have multiple systems with different cpu and ram each the cpu can be used to utilize 100% but i could not do it for the ram HDP-2.6.4.0

ilia987 · ‎07-08-2018

Thanks but i have done it already, still looking for better approach cause of maintenance needed every time new user is added

ilia987 · ‎06-27-2018

Workaround solution: I made sub queues for each user and it works, but you have to manage it for each user, (we are small company and have few users so it manageable) root.Dev.Dev_user1 root.Dev.Dev_user2 root.Dev.Dev_user3 root.Prod.Prod_user1 root.Prod.Prod_user2 root.Prod.Prod_user2 I configured this on our grid, but still looking for a better approach

ilia987 · ‎06-27-2018

I found a working dirty solution, added the queues as follow: root.bt.bt_user1 root.bt.bt_user2 root.bt.bt_user3 root.opt.opt_user1 root.opt.opt_user2 root.opt.opt_user3 and updated the submission script to match it

Online	Offline
Last Visited	‎08-01-2022 04:18 AM

Member Since	‎09-28-2017 09:04 AM
Last Visited	‎08-01-2022 04:18 AM
Posts	88
Kudos received	3

Cloudera Community

Host Metrics" section shows "No Data Available" o...

Re: yarn configure to use all ram

Re: yarn configure to use all ram

Re: yarn configure to use all ram

Re: yarn configure to use all ram

Re: yarn configure to use all ram

yarn configure to use all ram

Re: Yarn Capacity Scheduler: Share resource betwee...

Re: Yarn Capacity Scheduler: Share resource betwee...

Re: spark submit Exception unknown queue