Member since
01-10-2023
5
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2541 | 03-19-2024 10:33 AM |
06-26-2025
11:28 PM
Hello @VidyaSargur Can you please support me here to understand and fix this issue It's critical as end users are not able to query the tables
... View more
06-26-2025
11:04 PM
Problem statement: When i try to query iceberg table for current date which recieves data from streaming pipeline in interval of 5 mins Example: select * from <table> where result_date="<current_date>" limit 1; Error: ImpalaRuntimeException: Cannot find file in cache:: Cannot find file in cache: hdfs://xx/ya/Zzz/data/resulted/00004-22575-da5239e5-71d0-4b2f-af6b-73cbf4b7d9c5-46884-00001.parquet with snapshot id: 2154647205402518684 Workaround tried: Invalidate metadata or refresh - works for few mins until next commit occurs and then throws same error with new file and new snapshot id Tried setting below as tblproperties but no help ALTER TABLE db.table_name SET TBLPROPERTIES ( 'metadata_refresh_interval_ms' = '60000', 'refresh-before-read' = 'true' ); Even tried to understand whether below properties have any impact but seems like no write.metadata.delete-after-commit.enabled write.metadata.previous-versions-max unable to understand why this issue is poping where as iceberg maintains isolation. Where as same table can be queried via spark3-shell Also with same table properties some tables which gets data from same pipeline with same interval i am able to query successfully but not for few tables Any solution would be of great help.
... View more
Labels:
- Labels:
-
Apache Iceberg
-
Apache Impala
03-05-2024
06:01 AM
Hello @ChethanYM It works But we can set as default property in our cloudera manager for impala
... View more
01-10-2023
10:11 AM
Hello , I am facing issues while executing some queries , I am getting below error for time being and the issue is being resolved autoamtically . ERROR : File 'xxx/yyy/zzz/part-00000-c25df0cd-b7fa-4432-b416-16411dc390d4.c000.snappy.parquet' has an incompatible Parquet schema for column 'proc_mws.mws_xml.zeits'. Column type: TIMESTAMP, Parquet schema: optional byte_array charge [i:12 d:1 r:0] I checked the files but there is no schema mismatch . Can you please suggest me how i can fix the issue.
... View more
Labels:
- Labels:
-
Apache Impala