I managed
to manually get a second RS running on a host with the following changes: 1)
Configuration The
configuration of our first RS on this host lies within
/usr/hdp/current/hbase-regionserver/conf, which actually links to
/etc/hbase/2.5.3.0-37/0 . In order to
have a second configuration I copied the '0' folder to '1' in the same parent
folder (same permissions as the '0' folder) and adapted the following
parameters: within
hbase-site.xml hbase.regionserver.info.port from 16030 to 26130
hbase.regionserver.port from 16020 to 26020 within
hbase-env.sh export HBASE_CONF_DIR=${HBASE_CONF_DIR:-/etc/hbase/2.5.3.0-37/1}export HBASE_LOG_DIR=/var/log/hbase/hbase1export HBASE_PID_DIR=/var/run/hbase/hbase1 I changed
log and pid directory to a subfolder as the permissions to write are already
set correctly Adapting
the WAL directory the RS writes to is not necessary, as the port is contained
in the folder, and thus it's unique (I got the information from
https://amahanty.wordpress.com/2012/11/04/understanding-hbase-files-and-directories ) 2)
Execution Even though
there's only one hbase-regionserver "folder" within /usr/hdp/current
, this is sufficient, as we can define another configuration folder upon
startup Based on
the original command that is logged when restarting a RS via Ambari Execute['/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-regionserver/conf start regionserver'] {'not_if': 'ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-regionserver.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-regionserver.pid` >/dev/null 2>&1',
'user': 'hbase'} I executed
the following: /usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config /etc/hbase/2.5.3.0-37/1 start regionserver 3) Finally Within the
hbase-master web UI I saw the new RS, and after some time, the regions were
automatically distributed to the new RS. Within the
log file I see impl.MetricsSystemImpl: Error creating sink 'timeline'org.apache.hadoop.metrics2.impl.MetricsConfigException:
Error creating plugin: org.apache.hadoop.metrics2.sink.timeline.HadoopTimelineMetricsSink so probably
I'm missing some metrics configuration, but I haven't taken a look any further
at the moment. Probably ports? Anyhow, we
can't manage the new RS with Ambari, which makes sense, as Ambari doesn't know
anything about this service. Well, the
obvious question now is: How can we extend Ambari in order to install further
RS on one host? Open
questions: a) File
/usr/hdp/current/hbase-regionserver/conf/regionservers - When
adding an additional RS on a host, would it result in a second entry in the
regionservers file? - What's
the meaning of this file? b)
/etc/hbase/2.5.3.0-37/0 - the
folder '0', what's the meaning behind this 'zero' folder ? c) Does the
Ambari design support adding additional RS on one host? I cloned
the source from https://git-wip-us.apache.org/repos/asf/ambari.git and browsed
through the code, checked out the ambari design document etc. and have a basic
overview of the design, really basic (just to underline this) What I
think right now how it works, and please correct me if I'm wrong. Basically,
when I add a new host, the basic file structure for all services is put on this
host. Only when assigning a component to this newly added host, some magic
happens. With
'magic' I mean all the operations that need to be done in order to be able to
monitor the component with Ambari. At this
point I'm not sure where to add some code in order to define a different
configuration folder for the RS. Of course, I also want to be able to
restart/stop/start etc the component via Ambari, and when I make any changes to
the configuration of a RS or HBase itself, the additional RS should also be
considered. Can you
give me any hints? Thanks.
... View more
