Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

what does this mean? - ( overflow when multiplying cardinalities )

Solved Go to solution

what does this mean? - ( overflow when multiplying cardinalities )

Expert Contributor
W0105 16:59:22.911566 28206 PlanNode.java:545] overflow when multiplying cardinalities: 9223372036854775807, 9
W0105 16:59:22.909112 28206 PlanNode.java:545] overflow when multiplying cardinalities: 86038345052864512, 75227480
W0105 16:59:22.897722 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913
W0105 16:59:22.886450 28206 PlanNode.java:545] overflow when multiplying cardinalities: 2389953856178920, 12537913

Hell,

 

I started getting this message. How should I interpret this msg in log?

 

#udpated

I have found this from Google Group commented by Martin Grund.

"""

It happens when multiplying to Java long values results in an overflow, see https://github.com/cloudera/Impala/blob/cdh5-trunk/fe/src/main/java/com/cloudera/impala/planner/Plan....

 
Cardinalities are used in optimizing the plan and for example for joins table cardinalities are multiplied and may result in very larger values. If the multiplication overflows, we use Long.MAX_VALUE.

"""

 

#Question:

Is there anything I need to do to prevent this issue?

 

Thank you

Gatsby

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: what does this mean? - ( overflow when multiplying cardinalities )

Master Collaborator

At the moment you cannot really prevent this issue. It is a warning indicating that some calculations may not be accurate due to overflow (e.g., when multiplying very large cardinalities). The values will be capped at MAX_LONG.

 

If you don't have reason to believe that your query plans are bad because of this overflow, then I would not worry about those warnings.

2 REPLIES 2

Re: what does this mean? - ( overflow when multiplying cardinalities )

Master Collaborator

At the moment you cannot really prevent this issue. It is a warning indicating that some calculations may not be accurate due to overflow (e.g., when multiplying very large cardinalities). The values will be capped at MAX_LONG.

 

If you don't have reason to believe that your query plans are bad because of this overflow, then I would not worry about those warnings.

Highlighted

Re: what does this mean? - ( overflow when multiplying cardinalities )

Expert Contributor
Alex,

Thank you very much. :)



--
Moonwon (Gatsby) Lee
gatsbylee.com

"Life isn't about waiting for the storm to pass, it's about learning to
dance in the rain."
Don't have an account?
Coming from Hortonworks? Activate your account here