https://community.cloudera.com/t5/Archives-of-Support-Questions/What-is-serialization-in-ORC/m-p/224405#M75046 <P>Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.</P><P>See the following Wikipedia article for more detail: </P><P><A href="https://en.wikipedia.org/wiki/Serialization" target="_blank">https://en.wikipedia.org/wiki/Serialization</A></P> Tue, 27 Feb 2018 03:48:47 GMT anarasimham 2018-02-27T03:48:47Z https://community.cloudera.com/t5/Archives-of-Support-Questions/What-is-serialization-in-ORC/m-p/224404#M75045 <P>Link: <A href="https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC" target="_blank">https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC</A></P><P>Above link has a part named serialization. Can somebody tell what serialization is and for what it is used for?</P> Tue, 27 Feb 2018 02:24:55 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/What-is-serialization-in-ORC/m-p/224404#M75045 Hadoopy 2018-02-27T02:24:55Z https://community.cloudera.com/t5/Archives-of-Support-Questions/What-is-serialization-in-ORC/m-p/224405#M75046 <P>Serialization is the algorithm by which data is written to disk or transmitted somewhere. Different applications have different ways to serialize data to optimize for a specific outcome, whether it is dealing with reads or writes. As it says in the Hive language manual, integers and strings are encoded to disk and compressed in different ways, and it lists out the rules which it uses to do so. For example, variable-width encoding optimizes the space usage of the data, as it uses less space to encode smaller data.</P><P>See the following Wikipedia article for more detail: </P><P><A href="https://en.wikipedia.org/wiki/Serialization" target="_blank">https://en.wikipedia.org/wiki/Serialization</A></P> Tue, 27 Feb 2018 03:48:47 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/What-is-serialization-in-ORC/m-p/224405#M75046 anarasimham 2018-02-27T03:48:47Z