Reply
Expert Contributor
Posts: 205
Registered: ‎07-01-2015
Accepted Solution

Kudu web ui - cells read

Hi,

 I thought that during a particular scan Kudu is reporting a number of rows reand in realt-time per each column. At least on small table it was equal to roughly the number of rows in the partition.

 

But now I am scanning a 1 billion (1 000 000 000) row table, the table is partitioned into multiple partitions. And the cells read shows 2.3billion 4.6 billion etc.

 

 image.png

 

Can somebody explain why those numbers are so high?

 

Expert Contributor
Posts: 205
Registered: ‎07-01-2015

Re: Kudu web ui - cells read

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

Highlighted
Expert Contributor
Posts: 205
Registered: ‎07-01-2015

Re: Kudu web ui - cells read

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

Cloudera Employee
Posts: 64
Registered: ‎09-28-2015

Re: Kudu web ui - cells read

It looks like your screenshot is of the "scans" dashboard on the web UI.
This dashboard shows counters for a single scan, and a single scan would
only come from a single task, not aggregate across them.

I am guessing you're hitting KUDU-2231, a performance bug recently fixed.
The bug fix appears in CDH 5.14.0. Since this is a performance issue that
is not a regression and does not affect correctness, we have not yet
backported to any prior releases.

-Todd
Announcements