Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

what does this mean? - ( overflow when multiplying cardinalities )

Solved Go to solution

what does this mean? - ( overflow when multiplying cardinalities )

Contributor
W0105 16:59:22.911566 28206 PlanNode.java:545] overflow when multiplying cardinalities: 9223372036854775807, 9
W0105 16:59:22.909112 28206 PlanNode.java:545] overflow when multiplying cardinalities: 86038345052864512, 75227480
W0105 16:59:22.897722 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913
W0105 16:59:22.886450 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913

Hell,

 

I started getting this message. How should I interpret this msg in log?

 

#udpated

I have found this from Google Group commented by Martin Grund.

"""

It happens when multiplying to Java long values results in an overflow, see https://github.com/cloudera/Impala/blob/cdh5-trunk/fe/src/main/java/com/cloudera/impala/planner/Plan....

 
Cardinalities are used in optimizing the plan and for example for joins table cardinalities are multiplied and may result in very larger values. If the multiplication overflows, we use Long.MAX_VALUE.

"""

 

#Question:

Is there anything I need to do to prevent this issue?

 

Thank you

Gatsby

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: what does this mean? - ( overflow when multiplying cardinalities )

Master Collaborator

At the moment you cannot really prevent this issue. It is a warning indicating that some calculations may not be accurate due to overflow (e.g., when multiplying very large cardinalities). The values will be capped at MAX_LONG.

 

If you don't have reason to believe that your query plans are bad because of this overflow, then I would not worry about those warnings.

2 REPLIES 2

Re: what does this mean? - ( overflow when multiplying cardinalities )

Master Collaborator

At the moment you cannot really prevent this issue. It is a warning indicating that some calculations may not be accurate due to overflow (e.g., when multiplying very large cardinalities). The values will be capped at MAX_LONG.

 

If you don't have reason to believe that your query plans are bad because of this overflow, then I would not worry about those warnings.

Re: what does this mean? - ( overflow when multiplying cardinalities )

Contributor
Alex,

Thank you very much. :)



--
Moonwon (Gatsby) Lee
gatsbylee.com

"Life isn't about waiting for the storm to pass, it's about learning to
dance in the rain."