hbase.rootdir in Ambari Metrics configuration

[#1]

Deploying a single node cluster. For AMBARI_METRICS service, the hdfs.rootdir had a default value of "file:///local/var/lib/ambari-metrics-collector/hbase". This gave an error while configuring the services and suggested the name should be changed. I tried hdfs://hostname:8020/apps/ams/metrics. It gives the following warning:

In embedded mode hbase.rootdir cannot point to schemaless values or HDFS, Example - file:// for localFS Ambari Metrics service uses HBase as default storage backend. Set the rootdir for HBase to either local filesystem path if using Ambari Metrics in embedded mode or to a HDFS dir, example: hdfs://namenode.example.org:8020/amshbase.

Should I be giving the local folder for HDFS? And how can I find what folder this is?

[#2 · Accepted solution]

What version of Ambari is this?

I assume the value of your config item in ams-site : "timeline.metrics.service.operation.mode" is embedded.

Please verify the config ams-hbase-site : hbase.cluster.distributed = false.

If both of these configs are as stated above, and you still get an error for hbase.rootdir = file:///local/var/lib/ambari-metrics-collector/hbase, please omit the warning and go ahead the deploy the services.

Also, please share the screenshot for the warning you see so that we can create a bug to track it.

In general, AMS operates in 2 modes.

embedded : Single AMS HBase daemon writing to local disk.

distributed : Separate AMS HBase master and RS daemons by default writing to HDFS.

[#3]

This is Ambari 2.2.1.1

I dont see "timeline.metrics.service.operation.mode" under ams-site but it is under General. The value is embedded.

hbase.cluster.distributed = false.

Here is the warning message that I got.

[#4]

@Arvind Kandaswamy

As long as you have a folder called /local in the node where you are installing file:///local/var/lib/ambari-metrics-collector/hbase is a good value for hbase.rootdir.

You can ignore the error and proceed.

But please note that AMS might be writing loads of data - so it's always better to keep it away from datanode locations.

[#5]

If there is a local Datanode consider switching to AMS distributed mode, which means AMS will write to HDFS.

https://cwiki.apache.org/confluence/display/AMBARI/AMS+-+distributed+mode

Note: In distributed mode, HBase master needs much less memory vs RS, example say you have a 50 node cluster consider these memory settings:

ams-env :: collector_heapsize = 2048

ams-hbase-env :: hbase_master_heapsize = 512

ams-hbase-env :: hbase_master_xmn = 102

ams-hbase-env :: hbase_regionserver_heapsize = 4096

ams-hbase-env :: regionserver_xmn_size = 512

[#6]

Also, for reference, you can checkout this section of the docs related to Tuning Ambari Metrics (with some guidelines) and how to setup Distributed mode.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_ambari_reference_guide/content/ch_amb_ref...