- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to take a backup hive table (partitioned) in HDinsight cluster (blob storage)
- Labels:
-
Hortonworks Data Platform (HDP)
Created ‎12-22-2016 07:16 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi i am new to HDinsight cluster with HDP 2.4 , I just want to take a backup of file and remove some record one of hive table(partitoned) which is stored in blob storage(cloud)
i have done this in hive table which are stored in local hdfs
Create table bkp_table as select * from original_table;
so like above i done in HDinsight cluster
1) I have create database ' bkp_database' in hive(but which is not storage in blob storage)
2) i have take the backup of table
Create table bkp_database.bkp_table as select * from org_database.original_table(which is is stored in blob storage);
Here i observe that
1) table size is differ(blob storage have orc format)
2)table count() not matches
Can any one explain the step-step by process how to take a backup of hive table in hdsight cluster?
Created ‎12-22-2016 07:25 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
could you please try to
create table bkp_table like original_table;
and insert data into bkp_table from original table and see if it helps
Created ‎12-22-2016 07:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks you so much@Rajkumar Singh
Which database should i use to store this bkp_table i.e should i create any database?
If i create any database where should i create database in hive? or directory in blob storeage?
Created ‎12-22-2016 08:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
its upto you whether you want to create a table in new database or the same database where the original table is, for the second query you can specify database path aftter specifying path in command
create database <db name> location <some location>
Created ‎12-23-2016 01:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i have try to use "create table bkp_table like original_table;" and its created table same schema but location is local hdfs not in blob storage and format also not orc
Created ‎12-23-2016 05:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rama will you able to run the suggestions?
Created ‎12-23-2016 07:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
which suggestions?
i have run "create table bkp_table like original_table; but not use..it is stored in local hdfs not blob storage?
Created ‎12-23-2016 07:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
could you please try insert overwrite directory wasb:///<some location> select * from original table
then create a external table and point to this new location wasb:///<some location>
Created ‎12-23-2016 05:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have taken backup of table using below steps
1)Create one bkp directory in Blob storage
2) Create external backup table with schema like original table and location is bkp directory location in blob storage
3)insert the data using partition variable
now i observe that out of 3 files 2 files matches with row count(1) (i,e records are matches) and disk space little bit vary like 100 bytes(almost same)
but 3 table fully vary in size , here i need find the root cause..
Can you help how to restore my old drooped partition in hive ?
Created ‎12-25-2016 10:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
unfortunatly there is no way to restore partition in hive.
