Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kudu web ui - cells read

Solved Go to solution

Kudu web ui - cells read

Master Collaborator

Hi,

 I thought that during a particular scan Kudu is reporting a number of rows reand in realt-time per each column. At least on small table it was equal to roughly the number of rows in the partition.

 

But now I am scanning a 1 billion (1 000 000 000) row table, the table is partitioned into multiple partitions. And the cells read shows 2.3billion 4.6 billion etc.

 

 image.png

 

Can somebody explain why those numbers are so high?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kudu web ui - cells read

Master Collaborator

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

3 REPLIES 3

Re: Kudu web ui - cells read

Master Collaborator

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

Highlighted

Re: Kudu web ui - cells read

Expert Contributor
It looks like your screenshot is of the "scans" dashboard on the web UI.
This dashboard shows counters for a single scan, and a single scan would
only come from a single task, not aggregate across them.

I am guessing you're hitting KUDU-2231, a performance bug recently fixed.
The bug fix appears in CDH 5.14.0. Since this is a performance issue that
is not a regression and does not affect correctness, we have not yet
backported to any prior releases.

-Todd

Re: Kudu web ui - cells read

Master Collaborator

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.