question Re: Kudu web ui - cells read in Archives of Support Questions (Read Only)

Kudu web ui - cells read

Tomas79 — Fri, 16 Sep 2022 12:48:07 GMT

Hi,

I thought that during a particular scan Kudu is reporting a number of rows reand in realt-time per each column. At least on small table it was equal to roughly the number of rows in the partition.

But now I am scanning a 1 billion (1 000 000 000) row table, the table is partitioned into multiple partitions. And the cells read shows 2.3billion 4.6 billion etc.

Can somebody explain why those numbers are so high?

Re: Kudu web ui - cells read

Tomas79 — Wed, 31 Jan 2018 14:30:49 GMT

Found out that if multiple spark tasks are reading the same tablet (partition) then it counts multiple times the reads. Therefore the total cells read could be much higher than the number of rows in tablet, actual # of tasks x # rows.

Re: Kudu web ui - cells read

Tomas79 — Wed, 31 Jan 2018 14:31:06 GMT

Re: Kudu web ui - cells read

Todd Lipcon — Wed, 31 Jan 2018 18:05:17 GMT

It looks like your screenshot is of the "scans" dashboard on the web UI.
This dashboard shows counters for a single scan, and a single scan would
only come from a single task, not aggregate across them.

I am guessing you're hitting KUDU-2231, a performance bug recently fixed.
The bug fix appears in CDH 5.14.0. Since this is a performance issue that
is not a regression and does not affect correctness, we have not yet
backported to any prior releases.

-Todd