Support Questions
Find answers, ask questions, and share your expertise

Missing configurations in Cloudera Manager

Champion Alumni

Hello,

 

I wanted to folow the slides in order to optimize HBase :http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok 

 

However, there are some configurations that I didn't find in Cloudera Manager configurations (in their respective services):

 

  • namenode.avoid.read.stale.datanode = true
  • namenode.avoid.write.stale.datanode = true
  • namenode.stale.datanode.interval = 30000
  • client.read.shortcircuit.buffer.size = 131072
  • regionserver.checksum.verify = true
  • server.tcpnodelay = true
  • client.tcpnodelay = true
  • hregion.majorcompaction.jitter = 0.5 (½ week, default)
  • hstore.min.locality.to.skip.major.compact
  • master.wait.on.regionservers.timeout
  • ipc.client.tcpnodelay
  • zookeeper.useMulti

 

Why aren't these available? 

 

Thank you!

 

 

 

 

GHERMAN Alina
2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Missing configurations in Cloudera Manager

Super Collaborator

The below listed configuration appears to have no prefix such as dfs. hbase., by adding these I could see that they do exist in Cloudera Manager configuration. If there are some missing you can utilise the Custom Configuration [1] feature that CM provides for such cases.

 

Below are missing context, you should prefix them with dfs.

  • namenode.avoid.read.stale.datanode = true
  • namenode.avoid.write.stale.datanode = true
  • namenode.stale.datanode.interval = 30000
  • client.read.shortcircuit.buffer.size = 131072

 

  • regionserver.checksum.verify = true

When you upgrade to CDH 5, HBase checksums are now turned on by default. In CDH 4 you will have to modify HBASE safety-valve see [2]

 

This is context for core-site.xml, prefix with ipc. for HBASE use hbase. see [3], you will need to add these in the configuration safety-valve in the respective service.

  • server.tcpnodelay = true
  • client.tcpnodelay = true

 

These have also missing prefix hbase. see [4]

  • hregion.majorcompaction.jitter = 0.5 (½ week, default)
  • hstore.min.locality.to.skip.major.compact
  • master.wait.on.regionservers.timeout
  • ipc.client.tcpnodelay
  • zookeeper.useMulti [5]

 

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html

 

[2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hbase_new_features_and_changes...

 [3] http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html#_ipc_configuration_con...

[4] 

http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html

[5] CDH 5.5.1 comes with Zookeeper 3.4.5 (this is only applicable in conjuction of Zookeeper 3.5) 

 

 

View solution in original post

Re: Missing configurations in Cloudera Manager

Super Collaborator

> Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

Thing to note, Cloudera Manager team will update the configuration to match what is supported in CDH (HDFS, HBASE etc). If a parameter is not exposed in your CM it may be that your CM is dated or that property is not yet avalable/supported for the respective service. For example "dfs.namenode.avoid.read.stale.datanode" is exposed in CM 5.5.1 but not in CM 5.3.x

 

 

View solution in original post

3 REPLIES 3

Re: Missing configurations in Cloudera Manager

Super Collaborator

The below listed configuration appears to have no prefix such as dfs. hbase., by adding these I could see that they do exist in Cloudera Manager configuration. If there are some missing you can utilise the Custom Configuration [1] feature that CM provides for such cases.

 

Below are missing context, you should prefix them with dfs.

  • namenode.avoid.read.stale.datanode = true
  • namenode.avoid.write.stale.datanode = true
  • namenode.stale.datanode.interval = 30000
  • client.read.shortcircuit.buffer.size = 131072

 

  • regionserver.checksum.verify = true

When you upgrade to CDH 5, HBase checksums are now turned on by default. In CDH 4 you will have to modify HBASE safety-valve see [2]

 

This is context for core-site.xml, prefix with ipc. for HBASE use hbase. see [3], you will need to add these in the configuration safety-valve in the respective service.

  • server.tcpnodelay = true
  • client.tcpnodelay = true

 

These have also missing prefix hbase. see [4]

  • hregion.majorcompaction.jitter = 0.5 (½ week, default)
  • hstore.min.locality.to.skip.major.compact
  • master.wait.on.regionservers.timeout
  • ipc.client.tcpnodelay
  • zookeeper.useMulti [5]

 

[1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_config_snippet.html

 

[2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hbase_new_features_and_changes...

 [3] http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html#_ipc_configuration_con...

[4] 

http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-1.0.0-cdh5.5.1/book.html

[5] CDH 5.5.1 comes with Zookeeper 3.4.5 (this is only applicable in conjuction of Zookeeper 3.5) 

 

 

View solution in original post

Re: Missing configurations in Cloudera Manager

Champion Alumni

Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

However the safety valve ( Custom Configuration [1]) si a good ideea.

 

GHERMAN Alina

Re: Missing configurations in Cloudera Manager

Super Collaborator

> Most of them, I do not found them in CM even with the good prefixes (in their corresponding service).

Thing to note, Cloudera Manager team will update the configuration to match what is supported in CDH (HDFS, HBASE etc). If a parameter is not exposed in your CM it may be that your CM is dated or that property is not yet avalable/supported for the respective service. For example "dfs.namenode.avoid.read.stale.datanode" is exposed in CM 5.5.1 but not in CM 5.3.x

 

 

View solution in original post