Support Questions

Find answers, ask questions, and share your expertise

ORC Stripe size

avatar
Expert Contributor

I have orc.stripe.size=67108864

What size of stripe would be for small orc file (for example 2 Mb, 350K records)?

I thought that in this case file contains 1 stripe with size 2Mb. However in orcfiledump I see 426 stripes! Why?

1 ACCEPTED SOLUTION

avatar
Master Guru

That is very curious I have seen lots of stripes being created because of memory problems. But normally he only gets down to 5000 rows and then out of memory.

Which version of Hive are you using? What are your memory settings for the hive tasks and if the file is small is it possible that the table is partitioned and the task is writing into a large number of partitions at the same time?

Can you share the LOAD command and the table layout?

View solution in original post

10 REPLIES 10

avatar

Thanks Benjamin Leonhardi J,

Your slide 16 saved my day...