Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is serialization in ORC?

SOLVED Go to solution
Highlighted

What is serialization in ORC?

New Contributor

Link: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC

Above link has a part named serialization. Can somebody tell what serialization is and for what it is used for?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: What is serialization in ORC?

Expert Contributor

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization

1 REPLY 1

Re: What is serialization in ORC?

Expert Contributor

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization