Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Destination table is stored as ORC but the file being loaded is not a valid ORC file.

Destination table is stored as ORC but the file being loaded is not a valid ORC file.

New Contributor

org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException [Error 30019]: The file that you are trying to load does not match the file format of the destination table. Destination table is stored as ORC but the file being loaded is not a valid ORC file.

4 REPLIES 4
Highlighted

Re: Destination table is stored as ORC but the file being loaded is not a valid ORC file.

Contributor

How did you write the ORC file to this location (Pig, Spark, NiFi other?).

Can you show the schema of the table, the contents of the folder where ORC files are written, and also any details/code on how the file was ingested?

Re: Destination table is stored as ORC but the file being loaded is not a valid ORC file.

New Contributor

Thanks,http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

I am practicing from above link, but my hive.default.fileformat is ORC

find screen shots as wlel1.png2.png

Re: Destination table is stored as ORC but the file being loaded is not a valid ORC file.

New Contributor

http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

I am practicing from above link, but my hive.default.fileformat is ORC

1.png2.png

Re: Destination table is stored as ORC but the file being loaded is not a valid ORC file.

Contributor

Issues:

1) In your table definition "create table ..." you do not specify the LOCATION attribute of your table. Therefore Hive will default to look for the file in the default warehouse dir path. The location in your screenshot is under /user/admin/. You can run the command "show create table ..." to see where Hive thinks the table's files are located.

By default Hive creates managed tables, where files, metadata and statistics are managed by internal Hive processes. A managed table is stored under the hive.metastore.warehouse.dir path property, by default in a folder path similar to /apps/hive/warehouse/databasename.db/tablename/. The default location can be overridden by the location property during table creation.

2) You are specifying the format using hive.default.fileformat. I would avoid using this property. Instead simply use "STORED AS TEXTFILE" or "STORED AS ORC" in your table definition.

Please change the above, retest and let us know how that works

Don't have an account?
Coming from Hortonworks? Activate your account here