Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Kudu web ui - cells read

avatar

Hi,

 I thought that during a particular scan Kudu is reporting a number of rows reand in realt-time per each column. At least on small table it was equal to roughly the number of rows in the partition.

 

But now I am scanning a 1 billion (1 000 000 000) row table, the table is partitioned into multiple partitions. And the cells read shows 2.3billion 4.6 billion etc.

 

 image.png

 

Can somebody explain why those numbers are so high?

 

1 ACCEPTED SOLUTION

avatar

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

View solution in original post

3 REPLIES 3

avatar

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

avatar
Expert Contributor
It looks like your screenshot is of the "scans" dashboard on the web UI.
This dashboard shows counters for a single scan, and a single scan would
only come from a single task, not aggregate across them.

I am guessing you're hitting KUDU-2231, a performance bug recently fixed.
The bug fix appears in CDH 5.14.0. Since this is a performance issue that
is not a regression and does not affect correctness, we have not yet
backported to any prior releases.

-Todd

avatar

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.