Support Questions

Find answers, ask questions, and share your expertise

ANALYZE command not write data into hive metastore

avatar
Explorer

Good day,

In SIT, I have manually create a hive table. Follow by manually create a partition on it. Then I copy the bucket file from other table and put inside the partition. At this moment, if I run select statement, I getting 0 result, I believe this is normal because data haven't write to meta store.

Then I run the following commands:

ANALYZE TABLE raw_zone_sit.jxtemp PARTITION (dl_created_yr='2023', dl_created_mth='12') 
COMPUTE STATISTICS;

After I run this command, I able to select data from my table.

After that I apply the same way in production, but I am still getting empty result.

Currently I am trying to recover all the meta store in production, this is because 1 of the person accidently delete all the meta store in production. Thus now I am trying to recover on it.

For your information, parquet tables working fine. But I have some ORC tables also, now pending ORC table to resolve.

Please help.

1 ACCEPTED SOLUTION

avatar
Master Collaborator

@AhXian Ideally it is not supported to copy files into and out of an ACID table location. In this case we can use a workaround:

1. Create an external table similar(same columns) as the original ACID table.

2. Upload the data file into the external table location.

3. MSCK REPAIR table <external table name>

4. See if you can read the data from the external table now.

5. If [4] works, insert the data into the managed table by using:

insert into table managed_table_name partition(xx,yy) select * from external_table_name;

View solution in original post

14 REPLIES 14

avatar
Master Collaborator

Yes, 5 steps mentioned above

avatar
Explorer

Hi @smruti , Yes, I have done it in production environment, its not working, I still unable to select from it. But its working in SIT.

avatar
Master Collaborator

@AhXian 
1. Are you able to read data from the external table in Prod(step  4)?

2. In step 5, I meant for you to insert data into a new ACID table, which can be created using similar DDL as the original table.

3. After INSERT(step 5),  see if you could read the data from the new table.

Ideally it should work

avatar
Explorer

hi @smruti , I unable to read data from external table in step 4.
Thus, I cant use the insert data query to insert into a new ACID table in step 5.

avatar
Master Collaborator

@AhXian I hope we have copied the bucket files correctly, and also ran MSCK REPAIR. Could you also run ANALYZE.. COMPUTE STATISTICS on the external table once. Make sure the DESCRIBE command on the external table reflects the number of data files correctly. 

Please share describe and  "hdfs dfs -ls -R <externa table location>" o/p