Reply
New Contributor
Posts: 1
Registered: ‎03-22-2015

Cloudera installation dfs.datanode.max.locked.memory issue on LXC

I have created virtual box, ubuntu 14.04LTS environment on my mac machine.
In virtual box of ubuntu, I've created cluster of three lxc-containers. One for master and other two nodes for slaves.

On master, I have started installation of CDH5 using following link http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin

 

I have also made necessary changes in the /etc/hosts including FQDN and hostnames. Also created passwordless user named as "ubuntu".

 

While setting up the CDH5, during installation I'm constantly facing following error on datanodes. Max locked memory size: dfs.datanode.max.locked.memory of 922746880 bytes is more than the datanode's available RLIMIT_MEMLOCK ulimit of 65536 bytes.

 

Exception in secureMain: java.lang.RuntimeException: Cannot start datanode because the configured max locked memory size (dfs.datanode.max.locked.memory) of 922746880 bytes is more than the datanode's available RLIMIT_MEMLOCK ulimit of 65536 bytes.
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1050)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:411)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2297)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2184)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2231)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2407)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2431)

Cloudera Employee
Posts: 509
Registered: ‎07-30-2013

Re: Cloudera installation dfs.datanode.max.locked.memory issue on LXC

Hi,

What version of CM are you using? This sounds like a bug that was fixed in CM 5.3.1.

If you upgraded from CM4 to 5, you can also hit this error if you miss the step to hard restart CM agents, though I don't think that's relevant here.

Thanks,
Darren
Highlighted
Expert Contributor
Posts: 61
Registered: ‎03-06-2015

Re: Cloudera installation dfs.datanode.max.locked.memory issue on LXC

Try using

 

# wget http://archive-primary.cloudera.com/cm5/installer/5.3.2/cloudera-manager-installer.bin

 

# chmod +x cloudera-manager-installer.bin

 

# ./cloudera-manager-installer.bin

 

Should work better for you.

Announcements