Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Contributor

Introduction

Installing GeoMesa HBase and GeoServer on HDP

Prerequisites

  • HDP 2..6.4 on CentOS7.2
  • Java Development Kit 1.8,
  • Apache Maven,
  • a GitHub client,
  • and JAVA_HOME is configured

below commands can be used

yum install git -y

# configure JAVA_HOME
sed -i '$ a export JAVA_HOME=/usr/jdk64/jdk1.8.0_112' /etc/bashrc
sed -i '$ a export PATH=$PATH:$JAVA_HOME/bin' /etc/bashrc
source /etc/bashrc
java -version

#install Maven:
cd /tmp
wget http://mirror.jax.hugeserver.com/apache/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz
tar -zxvf apache-maven-3.5.3-bin.tar.gz

sudo mv /tmp/apache-maven-3.5.3 /opt
sudo chown -R root:root /opt/apache-maven-3.5.3
sudo ln -s /opt/apache-maven-3.5.3 /opt/apache-maven
echo 'export PATH=$PATH:/opt/apache-maven/bin' | sudo tee -a /etc/bashrc
source /etc/bashrc
mvn --version

Download source code

cd /tmp
git clone https://github.com/locationtech/geomesa.git
cd geomesa
# Build geomesa-hbase from source code
mvn clean install -pl geomesa-hbase/geomesa-hbase-dist -am
cd /tmp/geomesa/geomesa-hbase/geomesa-hbase-dist/target
cp geomesa-hbase_2.11-2.0.0-SNAPSHOT-bin.tar.gz /tmp
cd /tmp
tar -xvf geomesa-hbase_2.11-2.0.0-SNAPSHOT-bin.tar.gz
cp -R geomesa-hbase_2.11-2.0.0-SNAPSHOT /opt
ln -s /opt/geomesa-hbase_2.11-2.0.0-SNAPSHOT /opt/geomesa-hbase

Configure the environment to use an HDP install

Add the below into /etc/profile

sed -i '$ a export HADOOP_HOME=/usr/hdp/current/hadoop-client/' /etc/bashrc
sed -i '$ a export HBASE_HOME=/usr/hdp/current/hbase-client/' /etc/bashrc
sed -i '$ a export GEOMESA_HBASE_HOME=/opt/geomesa-hbase' /etc/bashrc
sed -i '$ a export PATH=$PATH:$GEOMESA_HBASE_HOME/bin' /etc/bashrc
source /etc/bashrc

Due to licensing restrictions, dependencies for shape file support must be separately installed.

Do this with the following commands:

cd /opt/geomesa-hbase
bin/install-jai.sh
bin/install-jline.sh
bin/install-saxon.sh

Deploying the GeoMesa HBase Distributed Runtime JAR

Setting the hbase.dynamic.jars.dir property in hbase-site.xml as custom property.

Make sure /apps/hbase/lib is already created in HDFS and change owner to hbase:hdfs

Copy GeoMesa-Hbase jar to HBase all Nodes

cp ${GEOMESA_HBASE_HOME}/dist/hbase/geomesa-hbase-distributed-runtime_2.11-2.0.0-SNAPSHOT.jar /usr/hdp/current/hbase-master/lib
chmod 777 /usr/hdp/current/hbase-master/lib/geomesa-hbase-distributed-runtime_2.11-2.0.0-SNAPSHOT.jar

Add proprety in Hbase-site.xml via Ambari

geomesa.hbase.remote.filtering=false
hbase.coprocessor.user.region.classes=org.locationtech.geomesa.hbase.coprocessor.GeoMesaCoprocessor

Modify zookeeper.znode.parent value in HBase-Site.xml (org: /hbase-unsecure --> New: /hbase)

zookeeper.znode.parent=/hbase

*Note: Make sure the HBase services are restarted *

Test the command that invokes the GeoMesa Tools:

$ bin/geomesa-hbase
INFO  Usage: geomesa-hbase [command] [command options]
  Commands:
  ...

Download and Build the Tutorial

cd ~/
git clone https://github.com/geomesa/geomesa-tutorials.git
cd geomesa-tutorials
mvn clean install -pl geomesa-tutorials-hbase/geomesa-tutorials-hbase-quickstart -am

Running the Tutorial

java -cp geomesa-tutorials-hbase/geomesa-tutorials-hbase-quickstart/target/geomesa-tutorials-hbase-quickstart-$VERSION.jar     org.geomesa.example.hbase.HBaseQuickStart     --hbase.zookeepers <zookeepers>               --hbase.catalog <table>
# example:

java -cp geomesa-tutorials-hbase/geomesa-tutorials-hbase-quickstart/target/geomesa-tutorials-hbase-quickstart-2.0.0-SNAPSHOT.jar     org.geomesa.example.hbase.HBaseQuickStart     --hbase.zookeepers pchalla2.field.hortonworks.com:2181,pchalla0.field.hortonworks.com:2181,pchalla1.field.hortonworks.com:2181     --hbase.catalog qs1

Verify the HBase

$ hbase shell
hbase(main):001:0> list
TABLE
qs1
qs1_gdelt_2dquickstart_attr_v4
qs1_gdelt_2dquickstart_id
qs1_gdelt_2dquickstart_z2_v2
qs1_gdelt_2dquickstart_z3_v2
5 row(s) in 0.3610 seconds

=> ["qs1", "qs1_gdelt_2dquickstart_attr_v4", "qs1_gdelt_2dquickstart_id", "qs1_gdelt_2dquickstart_z2_v2", "qs1_gdelt_2dquickstart_z3_v2"]

References:

http://www.geomesa.org/documentation/user/hbase/install.html

https://dev.locationtech.org/mhonarc/lists/geomesa-users/msg02028.html

https://viztales.wordpress.com/my-geomesa-experience/

Install GeoServer

wget http://sourceforge.net/projects/geoserver/files/GeoServer/2.12.2/geoserver-2.12.2-bin.zip
unzip geoserver-2.12.2-bin.zip
mv geoserver-2.12.2 /usr/share
cd /usr/share
ln -s geoserver-2.12.2 geoserver
echo "export GEOSERVER_HOME=/usr/share/geoserver" >> /etc/bashrc
source /etc/bashrc
# Check if GeoServer can be started successfully
cd geoserver/bin
sh startup.sh

open the GeoServer from post 8080/geoserver like below http://pchalla2.field.hortonworks.com:8080/geoserver

Installing GeoMesa HBase in GeoServer

The following JARs should be copied from the lib directory of your HBase or Hadoop installations into GeoServer’s WEB-INF/lib

cd /usr/share/geoserver/webapps/geoserver/WEB-INF/lib/
cp /opt/geomesa-hbase/dist/gs-plugins/geomesa-hbase-gs-plugin_2.11-2.0.0-SNAPSHOT-install.tar.gz .
tar -xvf geomesa-hbase-gs-plugin_2.11-2.0.0-SNAPSHOT-install.tar.gz
rm -f geomesa-hbase-gs-plugin_2.11-2.0.0-SNAPSHOT-install.tar.gz
cp /usr/hdp/2.6.4.0-91/hadoop//hadoop-annotations-2.7.3.2.6.4.0-91.jar .
cp /usr/hdp/2.6.4.0-91/hadoop//hadoop-auth-2.7.3.2.6.4.0-91.jar .
cp /usr/hdp/2.6.4.0-91/hadoop//hadoop-common-2.7.3.2.6.4.0-91.jar .
cp /usr/hdp/2.6.4.0-91/hadoop/lib/protobuf-java-2.5.0.jar .
cp /usr/hdp/2.6.4.0-91/hadoop/lib/commons-io-2.4.jar .
cp /usr/hdp/2.6.4.0-91/hbase/lib/hbase-server-1.1.2.2.6.4.0-91.jar .
cp /usr/hdp/2.6.4.0-91/hadoop/lib/zookeeper-3.4.6.2.6.4.0-91.jar .
cp /usr/hdp/2.6.4.0-91/hadoop/lib/commons-configuration-1.6.jar .
ln -s /usr/hdp/current/hbase-client/conf/hbase-site.xml ../classes/hbase-site.xml

Stop and Start the GeoServer to reflect the GeoMesa HBase plugin

Register the GeoMesa Store with GeoServer Log into GeoServer using your user and password credentials. The default administration credentials are: User name: admin; Password: geoserver

Click “Stores” and “Add new Store”. Select the HBase (GeoMesa) vector data source, and fill in the required parameters.

Basic store info:

workspace this is dependent upon your GeoServer installation data source name pick a sensible name, such as geomesa_quick_start description this is strictly decorative; GeoMesa quick start Connection parameters:

64911-nvds.png

these are the same parameter values that you supplied on the command line when you ran the tutorial; they describe how to connect to the HBase instance where your data reside Click “Save”, and GeoServer will search your HBase table for any GeoMesa-managed feature types.

Publish the Layer GeoServer should recognize the gdelt-quickstart feature type, and should present that as a layer that can be published. Click on the “Publish” link.

You will be taken to the “Edit Layer” screen. You will need to enter values for the data bounding boxes. In this case, you can click on the link to compute these values from the data. Bounding Boxes can be updated by clicking on "compute from data" or "compute from native bounds"

Click on the “Save” button when you are done.

Take a Look

Click on the “Layer Preview” link in the left-hand gutter. If you don’t see the quick-start layer on the first page of results, enter the name of the layer you just created into the search box, and press .

Once you see your layer, click on the “OpenLayers” link, which will open a new tab. You should see a collection of red dots similar to the following image:

64910-map.png

refer http://www.geomesa.org/documentation/tutorials/geomesa-quickstart-hbase.html

ref:

http://docs.geoserver.org/2.12.x/en/user/installation/linux.html https://github.com/geoserver/geoserver/tree/2.12.2

4,538 Views
Comments

I installed geomesa 1.3.5 in 10 node cluster. We are using kerberos to secure the cluster. I can able to ingest from local file system but I am not able to ingest csv data from HDFS to geomesahbase store, does geomesa works with kerberos?

Version history
Last update:
‎08-17-2019 08:04 AM
Updated by:
Contributors