question For ORC File what determines the number of mappers? in Support Questions

For ORC File what determines the number of mappers?

arun9a — Fri, 16 Sep 2022 09:54:47 GMT

For Orc file, how does yarn determine number of mapper. Is this based on the files in the hdfs?

aervits — Sun, 27 Dec 2015 12:31:18 GMT

OrcInputFormat is an implementation of InputFormat interface, where method getSplits determines the number of mappers https://hive.apache.org/javadocs/r0.13.1/api/ql/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.html.

bsuresh — Sun, 27 Dec 2015 14:25:56 GMT

@Aron,

Initially getsplits method splits the data based on the blocks in HDFS.But it was changed so that splitting is based on stripes of orc file.

The above link provides the complete link for details and source code for OrcInputformat and getSplit Method

arun9a — Sun, 27 Dec 2015 16:11:52 GMT

Thanks.