Code Repositories

Find and share code repositories
Labels (2)
Repo Description

This project shows how to analyze an HBase Snapshot using Spark.

The main motivation for writing this code is to reduce the impact on the HBase Region Servers while analyzing HBase records. By creating a snapshot of the HBase table, we can run Spark jobs against the snapshot, eliminating the impact to region servers and reducing the risk to operational systems.

At a high-level, here's what the code is doing:

  1. Reads an HBase Snapshot into a Spark
  2. Parses the HBase KeyValue to a Spark Dataframe
  3. Applies arbitrary data processing (timestamp and rowkey filtering)
  4. Saves the results back to an HBase (HFiles / KeyValue) format within HDFS, using HFileOutputFormat.
    • The output format maintains the original rowkey, timestamp, column family, qualifier, and value structure.
  5. From here, you can bulkload the HDFS file into HBase
Repo Info
Github Repo URL
Github account name zaratsian
Repo name SparkHBaseExample

Zaratsian i followed your tutorial but getting error "Wrong FS". can you help me solve this issue.

I have posted the question at below link

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.
Version history
Last update:
‎09-30-2016 12:02 AM
Updated by:
Top Kudoed Authors