Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SPARK job taking more memory then it is given

Highlighted

Re: SPARK job taking more memory then it is given

Rising Star

Spark Client is overriding the AM memory from 512mb to 896MB. Can you check Spark AM logs and see if the AM is overriding the container memory from 256MB to the higher value?

Highlighted

Re: SPARK job taking more memory then it is given

Contributor

Spark AM logs? Can you lead me please? :S

Highlighted

Re: SPARK job taking more memory then it is given

Contributor

@Gour Saha

Can you please help me?

Highlighted

Re: SPARK job taking more memory then it is given

Rising Star

In the RM UI, you can click on the app id link for a spark job and follow the app-attempt link and then click on the logs link against the first container (typically the one ending with 0001). Check the AM logs there and see what you find.

Don't have an account?
Coming from Hortonworks? Activate your account here