Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala can not read the files stored in the hive dir in hdfs

avatar
Contributor

Hi,

 

I have a table in Impala and it had some files in its location in hdfs. All of a sudden it can not read the files. I tried to invalidate and recover the partitions but nothing happened. I also moved the files from this folder into a new's folder path but it couldn't read them too.

 

What can I do ?

 

UPDATE

Tried to repair the table in hive

set hive.msck.repair.batch.size=1;
set hive.msck.path.validation=ignore;
msck REPAIR TABLE db.table

but received this error :

Error while processing statement: FAILED: Execution Error, return code 1 
from org.apache.hadoop.hive.ql.exec.DDLTask 

 

3 REPLIES 3

avatar
Super Collaborator

Hi,

 

Can you provide these below details to understand the issue:

 

1. Initially from where you created the table in hive or impala?

2. Provide the DDL of the table that you are trying to query from impala.

     # show create table <table-name>;

3. list the files from hdfs path ( you will see the table path in the DDL def. )

    # hdfs dfs -ls /table/path/

4. Are you able to access the same table data from hive?( if you are facing the issues from impala?  vice versa)

5. Go to HS2 logs/coordinator logs copy the entire error stack trace for the query that is failing and paste here.

 

I think these are the basic details needed for analysing the issue.

 

Regards,

Chethan YM

avatar
Contributor

Hi @ChethanYM ,

 

1)I have created in impala.

2)result
CREATE TABLE db.table (
...
)
PARTITIONED BY (
...
)
SORT BY (
...
)
STORED AS PARQUET
LOCATION 'hdfs://path/'
TBLPROPERTIES ('DO_NOT_UPDATE_STATS'='true', 'STATS_GENERATED'='TASK', 'impala.lastComputeStatsTime'='1627655409', 'numRows'='0', 'parquet.compression'='snappy', 'totalSize'='0')

3) I have listed the files and they seem ok

4)I can access it but returns again 0 results

5)It is not failing the query. It can not read the partitions as it did before.

 

 

 

 

avatar
Super Collaborator

Hi,

 

What is the query you are using to read the data from table? can you attach its "query profile" and coordinator logs to have a look?

 

Regards,

Chethan YM