hbase backup incremental on remote hdfs

[#1]

First I do a full backup with this command :

hbase backup create full hdfs://<hostname>:8020/apps/backup/hbase -all

No problem

When I do an incremental backup

hbase backup create incremental hdfs://<hostname>:8020/apps/backup/hbase -all

I obtain these error

Wrong FS: hdfs://<hostnme>:8020/apps/backup/backup/backup_1542105235709, expected: hdfs://hdcluster

With hostname=cbh530 or cbh530.bdxdom.mck it's the same result, already when I change the backup directory or the tables list and also when I use a backup set.

Can you help me please.

[#2]

Hi @ANTONIOU Thierry,

Please check under your HDFS configuration this property: dfs.internal.nameservices (it may be under Custom hdfs-site)

then do the full using it:

hbase backup create full hdfs://<nameservice>/apps/backup/hbase -all

and then the incremental

hbase backup create incremental hdfs://<nameservice>/apps/backup/hbase -all

In my case I tried the following:

hbase backup create full hdfs://hanamenode/tmp/backup -t emp

hbase backup create incremental hdfs://hanamenode/tmp/backup -t emp

Please let me know if you were able to run the incremental.

Regards,

AQ

[#3]

I try these syntax and obtain the error :

hbase@cbh790.bdxdom.mck@/home/hbase>hbase backup create full hdfs://DEV/apps/backup/hbase/POC-ARCHI -set all_tables
2018-11-15 11:45:37,621 ERROR [main] util.AbstractHBaseTool: Error running command-line tool
java.lang.IllegalArgumentException: java.net.UnknownHostException: DEV
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:438)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:321)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:690)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:631)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:160)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.hbase.backup.util.BackupClientUtil.checkPathExist(BackupClientUtil.java:72)
at org.apache.hadoop.hbase.backup.util.BackupClientUtil.checkTargetDir(BackupClientUtil.java:87)
at org.apache.hadoop.hbase.client.HBaseAdmin.backupTablesAsync(HBaseAdmin.java:2716)
at org.apache.hadoop.hbase.client.HBaseAdmin.backupTables(HBaseAdmin.java:2760)
at org.apache.hadoop.hbase.client.HBaseBackupAdmin.backupTables(HBaseBackupAdmin.java:243)
at org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:197)
at org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:111)
at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:126)
at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:131)
Caused by: java.net.UnknownHostException: DEV
... 23 more

[#4]

Hi @ANTONIOU Thierry

Please try:

hbase backup create full hdfs://hdcluster/tmp/backup -t emp

then

hbase backup create incremental hdfs://hdcluster/tmp/backup -t emp

Regards,

Ariel

[#5]

Sorry, but I don't want to backup on the same cluster, I want to backup cluster1 in a cluster2 hdfs directory .

[#6]

Hi,

sorry but I want to backup my hbase tables on HDFS directory on another cluster, these commands are allow to backup on the same cluster.

Did hbase backup broadcast the lan to search an HDP cluster by it name, I don't think so ?

I can do a FULL backup but not an incremental with the command

hbase backup create incremental hdfs://<Active Namenod>:8020/<HDFS Directory>

Thank's for your help.

[#7]

Hi @ANTONIOU Thierry,

Please open a case with Hortonworks support, there is a bug associated to this issue (EAR 9561)

Regards,

Ariel Quilodran

[#8]

Hi aquilodran,

thank's for this answer, I will look at.

Regards.