question How can we setup customized cgroups for Centos 7? in Archives of Support Questions (Read Only)

How can we setup customized cgroups for Centos 7?

prasad4u_com — Sat, 30 Jan 2016 02:47:09 GMT

I came to know that HDP is not supporting cgroups for CentOS 7. We are running our cluster on Centos 7. How can we enable CPU scheduling and Isolation? What is the process of defining our own cgroups for centos 7?

Re: How can we setup customized cgroups for Centos 7?

nsabharwal — Sat, 30 Jan 2016 02:51:04 GMT

@Ram D

Ram, You are right on the support. I suggest to wait until the official support before running into production.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/enabling_cgroups.html

Hint:

On RHEL7, the cpu and cpuacct controllers are managed together by default. The default directory is/sys/fs/cgroup/cpu,cpuacct. Due to the parsing logic of the container-executor, the presence of the comma in this path may lead to failures when initializing the NodeManager (when using the LinuxContainerExecutor).

To avoid this issue, create your own directory (such as /sys/fs/cgroup/hadoop/cpu) and set the yarn.nodemanager.linux-container-executor.cgroups.mount property to true. This will allow the NodeManager to mount the cpu controller, and YARN will be able to enforce CPU limits.

If you would like to mount CGroups yourself, you should set the yarn.nodemanager.linux-container-executor.cgroups.mount property to false and ensure that the hierarchy specified in the yarn.nodemanager.linux-container-executor.cgroups.hierarchy property exists in the mount location. You must also ensure that there are no commas anywhere in the path names.

Re: How can we setup customized cgroups for Centos 7?

prasad4u_com — Sat, 30 Jan 2016 03:05:10 GMT

Is there any need of cluster with kerberos security in this case?

Re: How can we setup customized cgroups for Centos 7?

nsabharwal — Sat, 30 Jan 2016 03:10:38 GMT

Yes afaik "CGroups require that the HDP cluster be Kerberos enabled."

Also, Kerberos is MUST in any case if we want to secure the cluster.

Re: How can we setup customized cgroups for Centos 7?

nsabharwal — Mon, 01 Feb 2016 22:39:28 GMT

@Ram D

See the following details. I would stick with official docs.