- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Please check that it is a valid Parquet file. This error can also occur due to stale metadata. If you believe this is a valid Parquet file, try running "refresh jmaster.hivetesttable".
- Labels:
-
Apache Impala
Created 02-14-2024 11:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I just installed a new Hadoop server which has hive and impala servers on it.
However, I am getting an error when try to import data from .cvs file from Impala shell:
The error is:
Created 02-14-2024 12:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@echodot Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our Impala experts @jAnshula @ezerihun who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Created 02-14-2024 03:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks @DianaTorres. @jAnshula @ezerihun Please let me know if you need more information.
Created 02-15-2024 01:46 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI @echodot
Did you try to execute refresh command on table --> jmaster.hivetesttable.
Does same error persist, after executing refresh command?
If the problem still persist, then execute below steps for confirming that the file is valid.
--> Download the partition or file in local disk.
--> Execute below command to validate the file
parquet-tools meta <full_path_to_file_name_in_the_local_disk>
Created 02-15-2024 01:02 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @jAnshula
I have run the refresh command on the table, but it did not help.
Here is the output after run the command parquet-tools meta
parquet-tools meta /user/hivetest/master_tables/hivetesttable/create_tables.sql
java.io.IOException: Could not read footer: java.lang.RuntimeException: file:/user/hivetest/master_tables/hivetesttable/create_tables.sql is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [10, 41, 59, 10]
Please suggest what I need to be done to fix the issue.
Thank you!
Created 02-16-2024 05:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @echodot
As the source table is not in parquet format, try ing below command to create and load the data
CREATE TABLE jmaster.hivetesttable AS SELECT * FROM JMASTER.ATABLE STORED AS PARQUET;
Created 02-16-2024 06:53 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @jAnshula
I created the table as parquet. However, I am still getting issue after insert data. Here is my query to create the table and insert to the table:
CREATE TABLE ATABLE
(
CHARCOL CHAR(10),
VCHARCOL VARCHAR(10),
DECIMALCOL DECIMAL(15,5),
NUMERICCOL DOUBLE,
SMALLCOL SMALLINT,
INTEGERCOL INT,
REALCOL FLOAT,
FLOATCOL FLOAT,
DOUBLECOL DOUBLE,
LVCOL STRING,
BITCOL BOOLEAN,
TINYINTCOL TINYINT,
BIGINTCOL BIGINT,
BINCOL STRING,
VARBINCOL STRING,
LVARBINCOL STRING,
DATECOL STRING,
TIMECOL STRING,
TSCOL TIMESTAMP
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS PARQUET;
Then insert data to the table: INSERT OVERWRITE TABLE ATABLE SELECT * FROM JMASTER.ATABLE
and the result:
Created 02-20-2024 06:39 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@jAnshula Hello, any help from my provided information above?
Created 02-20-2024 09:14 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try to execute the queries below and then run your query.
# alter table ATABLE set tblproperties('impala.disable.recursive.listing'='true');
# refresh ATABLE;
# check if your query works or not
Created 02-21-2024 05:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@jAnshulaI am getting below error when running your first query:
ERROR: AnalysisException: ALTER TABLE not supported on transactional (ACID) table: default.atable
