09-11-2014 07:09 AM
When setting the parameter "dfs.datanode.max.locked.memory" to a value other than 0, it is supposed to be the user's responsibility to set the MEMLOCK limits accordingly. CM does not leave that for the user and enforces it. (as seen on /proc/<pid>/limits)
This parameter is used by HDFS only to limt cache resources, so there is no real reason for CM to enforce this limit on an OS level. By doing so, CM does not allow external libraries (non-cache related) on the datanode to allocate memory past this limit.
To make a long story short, I'd like to cancel CM's enforcement of this limit and set the limits myself. Is there any way I could do so?
09-11-2014 07:24 AM