Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive with Avro Integration

Hive with Avro Integration

Contributor

I am new to Avro and any input on below clarifications is appreciated.

I got clarifications While reading Article on Avro with Hive.

Article Link

1)we are not mentioning .avsc schema file here.I read in some other article where they will be mentioning .avsc file in Table properties.Is it mandatory or optional?

hive> CREATE EXTERNAL TABLE user_profile (id BIGINT, name STRING, bday STRING) STORED AS avro; 

2)For the below change do I need to recreate the .avsc file each and every time?Is it one-time activity or for every step like a,b,c i need to recreate the .avsc file?

You can see that a number of operations can be allowed as a simple requirement change:

a)Adding a new column to a table (the “country” column in the 2nd file)

b)Dropping a column from a table (the “id” column in the 3nd file)

b)Renaming a column (the “birthday” column in the 4th file)

1 REPLY 1
Highlighted

Re: Hive with Avro Integration

Super Guru
@vamsi valiveti

Starting with Hive 0.14, you don't need to specify Avro schema. It can be interpreted by Hive (Avto Serde). Adding a new column is also supported by a simple "ALTER TABLE ADD COLUMN <column name type>". More details on this link.

Changing column name is simple since it only changes the metadata.

//change column a's name to a1.

ALTER TABLE test_change CHANGE a a1 INT;

Replace column lets you drop a column but I am not sure if its Avro is supported. According to the documentation in

link above, I don't see Replace supporting Avro.

Don't have an account?
Coming from Hortonworks? Activate your account here