Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cluster 2.6.5 is getting memory issues

Highlighted

Cluster 2.6.5 is getting memory issues

Explorer

cluster-memory.pngrunning-applications.pngI am a developer trying to help with a POC with the following configuration an 8 node 2.6.5 cluster

  • 1 edgenode (62 GB RAM 16 core)
  • 2 Namenodes (62 GB RAM 16 core & 12 GB RAM 4 core)
  • 5 datanodes (187 GB RAM 32 cores each)

my Cluster users are experiencing resource issues, I have 2 users running spark through Zeppelin notebook and it's clogging the cluster, literally, it consumes 93% of the resources. I have tried running the YARN Utility Script but I think I am getting mixed up. Based on the screenshots attached I am giving the following parameters the script I have hbase is installed, python yarn-utils.py -c 32 -m 187 -d 7 -k True https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_command-line-installation/content/determ... After the script has successfully run I change the Yarn and Mapred settings but I end up with only 11 cores what am I doing wrong?

What's the correct way of running the script taking into account the memory and cores available. How should I configure the spark environment not to use up all the memory or to release once the job is done?I just feel I am not doing the right thing.

1 REPLY 1

Re: Cluster 2.6.5 is getting memory issues

1. Spark Dynamic allocation

I believe your Zeppelin is configured to spawn as many executors as possible for SPARK. Kindly enable Dynamic allocation for Spark in Zeppelin.


2. Yarn Queue User Limit.
Can you also check whats your YARN queue configuration.
You can limit the number of containers that can be used by a given user using user limit factor.

Don't have an account?
Coming from Hortonworks? Activate your account here