question Impala Not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class in Support Questions

Impala Not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class

HareshAmin — Thu, 11 Nov 2021 09:25:33 GMT

Impala is not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class. Please suggest the solution.

Table aggregate_test is created using Hive and org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class.

Following query is not working:

SELECT * from open_aggregate_test.aggregate_test ;

Above query gives following error:

SQL Error [500051] [HY000]: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:AnalysisException: Failed to load metadata for table: 'open_aggregate_test.aggregate_test'
CAUSED BY: TableLoadingException: Could not load table open_aggregate_test.aggregate_test from catalog
CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[TableLoadingException: Failed to load metadata for table: open_aggregate_test.aggregate_test
CAUSED BY: InvalidStorageDescriptorException: Impala does not support tables of this type. REASON: SerDe library 'org.apache.hadoop.hive.serde2.OpenCSVSerde' is not supported.]), lookup_status:OK)
), Query: SELECT * from open_aggregate_test.aggregate_test.

Re: Impala Not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class

smruti — Sun, 14 Nov 2021 08:48:24 GMT

@HareshAmin As you correctly said, Impala does not support the mentioned OpenCSVSerde serde.

So, you could recreate the table using CTAS, with a storage format that is supported by both Hive and Impala.

CREATE TABLE new_table STORED AS PARQUET AS SELECT * FROM aggregate_test;

Re: Impala Not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class

HareshAmin — Tue, 16 Nov 2021 17:08:30 GMT

Thanks @smruti for your reply. I will check this option.

Re: Impala Not supporting org.apache.hadoop.hive.serde2.OpenCSVSerde SerDe class

VidyaSargur — Thu, 18 Nov 2021 05:18:16 GMT

@HareshAmin, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.