Created 09-11-2022 02:01 PM
Hi Team,
We are running some Hive, and Impala queries and getting the below error for the job.
It was running fine earlier, but now it's failing.
We tried multiple ways, like re-created the tables and then tested with the REFRESH command for the table.
Whenever the job executes then automatically permission is changed with the user. It was like impala: hive
Could you please suggest how we can w take it forward?
Note: We recently upgraded the cluster to CDP.
Created 09-14-2022 01:21 PM
@RamuAnnamalai It looks similar to https://issues.apache.org/jira/browse/IMPALA-10042
Please check what's the value of "Maximum Cached File Handles" under Impala Configuration in CM UI? Set that to Zero(0) and see if the issue still reappears.
How do you write to the table? Is there a chance, the data is getting corrupted during the insert?
Created 09-27-2022 06:25 AM
May i know if the table was created from the data that was exported in some other format like 'txt' format or something ? if this is true then, starting from CDP 7.x versions, the default file format is parquet. So, when the table is imported, it will be created in parquet format, but its original files will in 'txt' format.