Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

ORC Schema evolution in hive

New Contributor

How does schema evolution work in orc table in hive0.14? I can't seem to get it to respect adding column without breaking data already in the table. Is column type change int-> long supported in hive0.14?

4 REPLIES 4

Mentor

AFAIK, schema evolution for ORC is not available until Hive 2.0 https://issues.apache.org/jira/plugins/servlet/mobile#issue/HIVE-11981

Expert Contributor

@Artem Ervits Does "Alter Column Type" fall under Schema Evolution?

Mentor

@mmccline can you comment on that? Though from Matt's comments on the Jira, doubt it does

Schema evolution here is limited to adding new columns and a few cases of column type-widening (e.g. int to bigint).

Renaming columns, deleting column, moving columns and other schema evolution were not pursued due to lack of importance and lack of time. Also, it appears a much more sophisticated metadata would be needed to support them. @Saumil Mayani

Cloudera Employee

Actually, adding columns to the end of the table works from Hive 1.2 (via HDP 2.5.4).

In particular, you can use

alter table <tbl> add columns (<col1> <type1>, <col2> <type2>);

In Hive 2.1, you get additional abilities to change the column types. In the eventual Hive 2.2, you'll get the ability to delete and reorder columns.