I want to know how to configure docker containers resource management by YARN

[#1]

Hello, every one.

Currently, I try to use Presto in docker on YARN.

I have 3 node manager hosts and 2 resource maneger hosts (this is because resource manager is HA) in my HDP cluster.

I completed creating docker containers on all node manager hosts and was able to run Presto server (one of three is presto-coordinator and remain are presto-workers) by using overlay network.

Also, I know that YARN supports docker containers and can manage docker containers resource (for example: CPU Usages).

Now, in my above use case, I want to know how to configure docker containers resource management by YARN.

Thank you.

[#2 · Accepted solution]

@takuma

what version of hdp are you in ? docker support for yarn is added in HDP 3.0 below are the links with steps to configure the same.

https://hortonworks.com/blog/part-5-of-data-lake-3-0-yarn-and-containerization-supporting-docker-and...

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/data-operating-system/content/run_docker_co...

https://community.hortonworks.com/content/kbentry/226331/dockerized-yarn-services-quickstart.html

Cheers,

Naveen

[#3]

@Naveen

Thank you for answering my question.

HDP version in my environment is 3.0.1.

Also, I appreciate for your sending the link about configuring docker on yarn.

I want to try it.

Thanks.