Missing configurations in Cloudera Manager

[#1]

Hello,

I wanted to folow the slides in order to optimize HBase :http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok 

However, there are some configurations that I didn't find in Cloudera Manager configurations (in their respective services):

• namenode.avoid.read.stale.datanode = true
• namenode.avoid.write.stale.datanode = true
• namenode.stale.datanode.interval = 30000
• client.read.shortcircuit.buffer.size = 131072
• regionserver.checksum.verify = true
• server.tcpnodelay = true
• client.tcpnodelay = true
• hregion.majorcompaction.jitter = 0.5 (½ week, default)
• hstore.min.locality.to.skip.major.compact
• master.wait.on.regionservers.timeout
• ipc.client.tcpnodelay
• zookeeper.useMulti

Why aren't these available? 

Thank you!

[#2 · Accepted solution]

The below listed configuration appears to have no prefix such as dfs. hbase., by adding these I could see that they do exist in Cloudera Manager configuration. If there are some missing you can utilise the Custom Configuration [1] feature that CM provides for such cases.

Below are missing context, you should prefix them with dfs.

• namenode.avoid.read.stale.datanode = true
• namenode.avoid.write.stale.datanode = true
• namenode.stale.datanode.interval = 30000
• client.read.shortcircuit.buffer.size = 131072

• regionserver.checksum.verify = true

When you upgrade to CDH 5, HBase checksums are now turned on by default. In CDH 4 you will have to modify HBASE safety-valve see [2]

This is context for core-site.xml, prefix with ipc. for HBASE use hbase. see [3], you will need to add these in the configuration safety-valve in the respective service.

• server.tcpnodelay = true
• client.tcpnodelay = true

These have also missing prefix hbase. see [4]

• hregion.majorcompaction.jitter = 0.5 (½ week, default)
• hstore.min.locality.to.skip.major.compact

• master.wait.on.regionservers.timeout
• ipc.client.tcpnodelay
• zookeeper.useMulti [5]

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html

[2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hbase_new_features_and_changes...

 [3] http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html#_ipc_configuration_con...

[4] 

http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html

[5] CDH 5.5.1 comes with Zookeeper 3.4.5 (this is only applicable in conjuction of Zookeeper 3.5) 

[#3 · Accepted solution]

The below listed configuration appears to have no prefix such as dfs. hbase., by adding these I could see that they do exist in Cloudera Manager configuration. If there are some missing you can utilise the Custom Configuration [1] feature that CM provides for such cases.

Below are missing context, you should prefix them with dfs.

• namenode.avoid.read.stale.datanode = true
• namenode.avoid.write.stale.datanode = true
• namenode.stale.datanode.interval = 30000
• client.read.shortcircuit.buffer.size = 131072

• regionserver.checksum.verify = true

When you upgrade to CDH 5, HBase checksums are now turned on by default. In CDH 4 you will have to modify HBASE safety-valve see [2]

This is context for core-site.xml, prefix with ipc. for HBASE use hbase. see [3], you will need to add these in the configuration safety-valve in the respective service.

• server.tcpnodelay = true
• client.tcpnodelay = true

These have also missing prefix hbase. see [4]

• hregion.majorcompaction.jitter = 0.5 (½ week, default)
• hstore.min.locality.to.skip.major.compact

• master.wait.on.regionservers.timeout
• ipc.client.tcpnodelay
• zookeeper.useMulti [5]

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html

[2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hbase_new_features_and_changes...

 [3] http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html#_ipc_configuration_con...

[4] 

http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html

[5] CDH 5.5.1 comes with Zookeeper 3.4.5 (this is only applicable in conjuction of Zookeeper 3.5) 

[#4]

Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

However the safety valve ( Custom Configuration [1]) si a good ideea.

[#5 · Accepted solution]

> Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

Thing to note, Cloudera Manager team will update the configuration to match what is supported in CDH (HDFS, HBASE etc). If a parameter is not exposed in your CM it may be that your CM is dated or that property is not yet avalable/supported for the respective service. For example "dfs.namenode.avoid.read.stale.datanode" is exposed in CM 5.5.1 but not in CM 5.3.x