Support Questions

Find answers, ask questions, and share your expertise

How to take a backup hive table (partitioned) in HDinsight cluster (blob storage)

avatar
Expert Contributor

Hi i am new to HDinsight cluster with HDP 2.4 , I just want to take a backup of file and remove some record one of hive table(partitoned) which is stored in blob storage(cloud)

i have done this in hive table which are stored in local hdfs

Create table bkp_table as select * from original_table;

so like above i done in HDinsight cluster

1) I have create database ' bkp_database' in hive(but which is not storage in blob storage)

2) i have take the backup of table

Create table bkp_database.bkp_table as select * from org_database.original_table(which is is stored in blob storage);

Here i observe that

1) table size is differ(blob storage have orc format)

2)table count() not matches

Can any one explain the step-step by process how to take a backup of hive table in hdsight cluster?

10 REPLIES 10

avatar
Super Guru

could you please try to

create table bkp_table like original_table;

and insert data into bkp_table from original table and see if it helps

@rama

avatar
Expert Contributor

Thanks you so much@Rajkumar Singh

Which database should i use to store this bkp_table i.e should i create any database?

If i create any database where should i create database in hive? or directory in blob storeage?

avatar
Super Guru
@rama

its upto you whether you want to create a table in new database or the same database where the original table is, for the second query you can specify database path aftter specifying path in command

create database <db name> location <some location>

avatar
Expert Contributor

@Rajkumar Singh

i have try to use "create table bkp_table like original_table;" and its created table same schema but location is local hdfs not in blob storage and format also not orc

avatar
Super Guru

@rama will you able to run the suggestions?

avatar
Expert Contributor
@Rajkumar Singh

which suggestions?

i have run "create table bkp_table like original_table; but not use..it is stored in local hdfs not blob storage?

avatar
Super Guru

@rama

could you please try insert overwrite directory wasb:///<some location> select * from original table

then create a external table and point to this new location wasb:///<some location>

avatar
Expert Contributor

@ Rajkumar Singh

I have taken backup of table using below steps

1)Create one bkp directory in Blob storage

2) Create external backup table with schema like original table and location is bkp directory location in blob storage

3)insert the data using partition variable

now i observe that out of 3 files 2 files matches with row count(1) (i,e records are matches) and disk space little bit vary like 100 bytes(almost same)

but 3 table fully vary in size , here i need find the root cause..

@ Rajkumar Singh

Can you help how to restore my old drooped partition in hive ?

avatar
Super Guru

@rama

unfortunatly there is no way to restore partition in hive.