what does this mean? - ( overflow when multiplying cardinalities )

[#1]

W0105 16:59:22.911566 28206 PlanNode.java:545] overflow when multiplying cardinalities: 9223372036854775807, 9
W0105 16:59:22.909112 28206 PlanNode.java:545] overflow when multiplying cardinalities: 86038345052864512, 75227480
W0105 16:59:22.897722 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913
W0105 16:59:22.886450 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913

Hell,

I started getting this message. How should I interpret this msg in log?

#udpated

I have found this from Google Group commented by Martin Grund.

"""

It happens when multiplying to Java long values results in an overflow, see https://github.com/cloudera/Impala/blob/cdh5-trunk/fe/src/main/java/com/cloudera/impala/planner/Plan....

Cardinalities are used in optimizing the plan and for example for joins table cardinalities are multiplied and may result in very larger values. If the multiplication overflows, we use Long.MAX_VALUE.

"""

#Question:

Is there anything I need to do to prevent this issue?

Thank you

Gatsby

[#2 · Accepted solution]

At the moment you cannot really prevent this issue. It is a warning indicating that some calculations may not be accurate due to overflow (e.g., when multiplying very large cardinalities). The values will be capped at MAX_LONG.

If you don't have reason to believe that your query plans are bad because of this overflow, then I would not worry about those warnings.

[#3]

Alex,

Thank you very much. 🙂



--
Moonwon (Gatsby) Lee
gatsbylee.com

"Life isn't about waiting for the storm to pass, it's about learning to
dance in the rain."