Support Questions
Find answers, ask questions, and share your expertise

how to recover hbase using hdfs data directory

Solved Go to solution
Highlighted

Re: how to recover hbase using hdfs data directory

Rising Star

Hi @Victor Xu,

Thanks. I understand your point.

I have couple of questions here to understand the scenario more clearly-

1. If I put data in temporary hbase cluster during main hbase cluster downtime, then how I will merge data from temporary cluster to main cluster when main cluster will be up and running.

2. When I am restoring data from hdfs hfile location to new location, then how I will recover memstore data.

3. If I shutdown restart hbase service, is memstore data being flushed to hdfs hfile that time?

Thanks,

Raja

Re: how to recover hbase using hdfs data directory

Contributor

Hi @Raja Ray,

To answer your questions:

1. If I put data in temporary hbase cluster during main hbase cluster downtime, then how I will merge data from temporary cluster to main cluster when main cluster will be up and running.

  • If there are only Put operations during the main cluster downtime, you can use CopyTable tool or Export& Bulkload tool to migrate data from temporary cluster back to main cluster after it's up.
  • But if there are both Put and Delete operations during the main cluster downtime, the best way to migrate data is to set up hbase replication from temporary cluster to main cluster. This will read all WALs(Write-ahead-log) and replay both Puts and Deletes on the main cluster after it's up.

2. When I am restoring data from hdfs hfile location to new location, then how I will recover memstore data.

  • Memstore is a place in RS to keep incoming data. It will start growing when new write operations are coming.
  • If you mean the blockcache of the hfile, that will be reload into memory when new read operations are coming.

3. If I shutdown restart hbase service, is memstore data being flushed to hdfs hfile that time?

  • Yes, memstore would be forced to flush to hfile before RS is shutdown.
  • Make sure hdfs path '/apps/hbase/data/WALs/' is empty after hbase being shutdown, so that all memstore data has been flushed into hfiles.

Thanks,

Victor

Highlighted

Re: how to recover hbase using hdfs data directory

Rising Star

Thanks a lot @Victor Xu.

All points are clear.