I looked at an impala query profile and found the "AverageThreadTokens" value is super large:
e.g., TCounter(name:AverageThreadTokens, type:DOUBLE_VALUE, value:4607182418800017408)
however, on the CM impala query page, I'm seeing only a reasonable number:
- AverageThreadTokens: 6.35
Can someone please explain why the query profile has such a big number? And how could I convert that into the reasonable numbers?
Thanks in advance!
How are you seeing that profile value? That's the raw value stored in the internal profile representation. DOUBLE values are encoded. The big number is just an encoding of 6.35.
I got the large value from the raw query profile which I downloaded from CM and Impalad UI. How could I convert/decode it to the reasonable "6.35" value?
You mean the thrift-encoded profiles? The text profiles I think should already display it in the correct way.
You need to reinterpret the bits of the 64-bit integer values as a double-precision floating point value. E.g. the C++ code in Impala that does this is here:
The logic is roughly:
long value = ...; double output = *reinterpret_cast<double*>(&value);