Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ORC Schema evolution in hive

Highlighted

ORC Schema evolution in hive

New Contributor

How does schema evolution work in orc table in hive0.14? I can't seem to get it to respect adding column without breaking data already in the table. Is column type change int-> long supported in hive0.14?

4 REPLIES 4

Re: ORC Schema evolution in hive

Mentor

AFAIK, schema evolution for ORC is not available until Hive 2.0 https://issues.apache.org/jira/plugins/servlet/mobile#issue/HIVE-11981

Re: ORC Schema evolution in hive

Expert Contributor

@Artem Ervits Does "Alter Column Type" fall under Schema Evolution?

Re: ORC Schema evolution in hive

Mentor

@mmccline can you comment on that? Though from Matt's comments on the Jira, doubt it does

Schema evolution here is limited to adding new columns and a few cases of column type-widening (e.g. int to bigint).

Renaming columns, deleting column, moving columns and other schema evolution were not pursued due to lack of importance and lack of time. Also, it appears a much more sophisticated metadata would be needed to support them. @Saumil Mayani

Re: ORC Schema evolution in hive

Cloudera Employee

Actually, adding columns to the end of the table works from Hive 1.2 (via HDP 2.5.4).

In particular, you can use

alter table <tbl> add columns (<col1> <type1>, <col2> <type2>);

In Hive 2.1, you get additional abilities to change the column types. In the eventual Hive 2.2, you'll get the ability to delete and reorder columns.

Don't have an account?
Coming from Hortonworks? Activate your account here