Reply
Explorer
Posts: 13
Registered: ‎08-07-2017

Large AverageThreadTokens value in impala's query profile

Hi,

 

I looked at an impala query profile and found the "AverageThreadTokens" value is super large:

 

e.g., TCounter(name:AverageThreadTokens, type:DOUBLE_VALUE, value:4607182418800017408)

 

however, on the CM impala query page, I'm seeing only a reasonable number:

 

- AverageThreadTokens: 6.35

 

Can someone please explain why the query profile has such a big number? And how could I convert that into the reasonable numbers?

 

Thanks in advance!

 

Cloudera Employee
Posts: 268
Registered: ‎07-29-2015

Re: Large AverageThreadTokens value in impala's query profile

How are you seeing that profile value? That's the raw value stored in the internal profile representation. DOUBLE values are encoded. The big number is just an encoding of 6.35.

Explorer
Posts: 13
Registered: ‎08-07-2017

Re: Large AverageThreadTokens value in impala's query profile

I got the large value from the raw query profile which I downloaded from CM and Impalad UI. How could I convert/decode it to the reasonable "6.35" value?

 

Thanks,

Cloudera Employee
Posts: 268
Registered: ‎07-29-2015

Re: Large AverageThreadTokens value in impala's query profile

You mean the thrift-encoded profiles? The text profiles I think should already display it in the correct way.

 

You need to reinterpret the bits of the 64-bit integer values as a double-precision floating point value. E.g. the C++ code in Impala that does this is here:

https://github.com/apache/incubator-impala/blob/d0d0af27863370d66a9f5c7fa902e08eac0a37d5/be/src/util...

 

The logic is roughly:

 

        long value = ...;

        double output = *reinterpret_cast<double*>(&value);
Cloudera Employee
Posts: 268
Registered: ‎07-29-2015

Re: Large AverageThreadTokens value in impala's query profile

Explorer
Posts: 13
Registered: ‎08-07-2017

Re: Large AverageThreadTokens value in impala's query profile

Thanks Tim! That's exactly what I was looking for.

 

Regards,

Announcements