question Re: Missing configurations in Cloudera Manager in Support Questions

Missing configurations in Cloudera Manager

AlinaGHERMAN — Fri, 16 Sep 2022 09:58:55 GMT

Hello,

I wanted to folow the slides in order to optimize HBase :http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok

However, there are some configurations that I didn't find in Cloudera Manager configurations (in their respective services):

namenode.avoid.read.stale.datanode = true
namenode.avoid.write.stale.datanode = true
namenode.stale.datanode.interval = 30000
client.read.shortcircuit.buffer.size = 131072
regionserver.checksum.verify = true
server.tcpnodelay = true
client.tcpnodelay = true
hregion.majorcompaction.jitter = 0.5 (½ week, default)
hstore.min.locality.to.skip.major.compact
master.wait.on.regionservers.timeout
ipc.client.tcpnodelay
zookeeper.useMulti

Why aren't these available?

Thank you!

Re: Missing configurations in Cloudera Manager

michalis — Fri, 22 Jan 2016 13:26:45 GMT

The below listed configuration appears to have no prefix such as dfs. hbase., by adding these I could see that they do exist in Cloudera Manager configuration. If there are some missing you can utilise the Custom Configuration [1] feature that CM provides for such cases.

Below are missing context, you should prefix them with dfs.

namenode.avoid.read.stale.datanode = true
namenode.avoid.write.stale.datanode = true
namenode.stale.datanode.interval = 30000
client.read.shortcircuit.buffer.size = 131072

regionserver.checksum.verify = true

When you upgrade to CDH 5, HBase checksums are now turned on by default. In CDH 4 you will have to modify HBASE safety-valve see [2]

This is context for core-site.xml, prefix with ipc. for HBASE use hbase. see [3], you will need to add these in the configuration safety-valve in the respective service.

server.tcpnodelay = true
client.tcpnodelay = true

These have also missing prefix hbase. see [4]

hregion.majorcompaction.jitter = 0.5 (½ week, default)
hstore.min.locality.to.skip.major.compact

master.wait.on.regionservers.timeout
ipc.client.tcpnodelay
zookeeper.useMulti [5]

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html

[2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hbase_new_features_and_changes.html

[3] http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html#_ipc_configuration_conflicts_with_hadoop

[4]

http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html

[5] CDH 5.5.1 comes with Zookeeper 3.4.5 (this is only applicable in conjuction of Zookeeper 3.5)

Re: Missing configurations in Cloudera Manager

AlinaGHERMAN — Fri, 22 Jan 2016 12:52:28 GMT

Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

However the safety valve ( Custom Configuration [1]) si a good ideea.

Re: Missing configurations in Cloudera Manager

michalis — Fri, 22 Jan 2016 13:25:21 GMT

> Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

Thing to note, Cloudera Manager team will update the configuration to match what is supported in CDH (HDFS, HBASE etc). If a parameter is not exposed in your CM it may be that your CM is dated or that property is not yet avalable/supported for the respective service. For example "dfs.namenode.avoid.read.stale.datanode" is exposed in CM 5.5.1 but not in CM 5.3.x