Support Questions
Find answers, ask questions, and share your expertise

What is serialization in ORC?

Explorer

Link: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC

Above link has a part named serialization. Can somebody tell what serialization is and for what it is used for?

1 ACCEPTED SOLUTION

Accepted Solutions

Expert Contributor

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization

View solution in original post

1 REPLY 1

Expert Contributor

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization

View solution in original post