Created 05-29-2017 08:49 AM
Ex. consider file has only 1 line. File size is 500MB. how hdfs stores the file? how many blocks are used to store such file in hdfs?
Created 05-29-2017 09:01 AM
Here is a Blog explaining how 512MB file is splitted in HDFS:
Created 05-29-2017 09:22 AM
My question is more around on line size.
Consider, a hive table has two rows, each with size of 500MB (more than Normal hdfs Block size).
How hdfs stores file when row size is greater than block size?
Does hdfs stores partial row in one block and remaining on other block.?
Created 05-29-2017 11:07 AM
Yes, HDFS partial row in one block and remaining on other block in the case where each row contains data that is more than the block size. Refer to the below link for more details:-