The best way to observe and run custom logic when a table is modified would be to write a master coprocessor. Your MasterObserver will be called before and after any table is modified (see https://hbase.apache.org/book.html#cp). However, unlike column families, columns in HBase are not predefined. So you cannot observe column changes since there is no fixed "schema". Please read up on the HBase data model. ... View more

TableInputFormat used in HBase will create 1 map task per table region. The data size will depend on how big your regions are. ... View more

Scans in HBase work in batches since there is no streaming RPC in HBase. A scanner is opened for a region and the scan is executed as a series of RPC calls to fetch the next set of results. Every such call is a "next" operation, referred as ScanNext. The scan next call tries to fetch either a predefined set of rows (scanner caching) or predefined max result size (2MB, etc). The behavior depends on the version of HBase as well as configuration. More info here: https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1 Seeing 10-90 seconds in the latency metrics means that most of the RPC call to get the next scan results ended up taking that long. It maybe due to a case where the scan is scanning a lot of data with a highly selective filter and not returning data or something else is wrong causing excessive latency for the scans. ... View more

The regionserver is failing because of this: 2016-10-0703:01:03,102 WARN [master/imp1tvhdpmst1.corp.test.com/172.24.125.130:16000] util.Sleeper:We slept 63817ms instead of 3000ms,thisis likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired Please check your GC settings and tune the GC. ... View more

Ambari does not use these files for managing the HBase service. Those files are only used for custom scripts that come with HBase (bin/start-hbase.sh, etc) which is not needed in an Ambari setup. ... View more

You are probably running into https://issues.apache.org/jira/browse/HBASE-1514. In secure clusters, you should run the zkcli command with --auth-as-server parameter. Like this: hbase --auth-as-server zkcli In newer versions of HDP, the patch should be there. Otherwise, you can get this working with something like: HBASE_OPTS="$HBASE_OPTS -Djava.security.auth.login.config=/usr/hdp/current/hbase-regionserver/conf/hbase_master_jaas.conf hbase zkcli Please note that the jaas.conf file that you add to the HBASE_OPTS before running the zkcli command should match what you have for your master and regionservers. ... View more