Support Questions

Find answers, ask questions, and share your expertise

What is serialization in ORC?

avatar
Contributor

Link: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC

Above link has a part named serialization. Can somebody tell what serialization is and for what it is used for?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization

View solution in original post

1 REPLY 1

avatar
Super Collaborator

Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.

See the following Wikipedia article for more detail:

https://en.wikipedia.org/wiki/Serialization