Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to determine number of mapper w/o knowing the block size

Highlighted

How to determine number of mapper w/o knowing the block size

Explorer

Hi,

 

How to determine number of mapper w/o knowing the block size. Only we have 2 input files , but dont know the exact size of the input file.

Can anyone tell me can we determine the same?

1 REPLY 1

Re: How to determine number of mapper w/o knowing the block size

Master Collaborator

The number of mappers should depend on how many disks your data is spread across, so IMO it's more a question of how big your cluster is. The block size and file size (and the replication factor) determine how many blocks there are, but it's really how many tasks can concurrently access blocks that you should be asking yourself.