For ORC File what determines the number of mappers?

[#1]

For Orc file, how does yarn determine number of mapper. Is this based on the files in the hdfs?

[#2]

OrcInputFormat is an implementation of InputFormat interface, where method getSplits determines the number of mappers https://hive.apache.org/javadocs/r0.13.1/api/ql/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.html.

[#3 · Accepted solution]

@Aron,

Initially getsplits method splits the data based on the blocks in HDFS.But it was changed so that splitting is based on stripes of orc file.

https://issues.apache.org/jira/browse/HIVE-5102

The above link provides the complete link for details and source code for OrcInputformat and getSplit Method

[#4]

Thanks.

http://mail-archives.apache.org/mod_mbox/hive-user...