Community
Training
Partners
Support

Cloudera Community

Cloudera Community
Announcements
Using the Community
Support
- Support Questions
- Community Articles
Groups
- Cloudera Data Analytics (CDA)

Support Questions

Find answers, ask questions, and share your expertise

cancel

Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Show only | Search instead for

Did you mean:

Advanced Search

Cloudera Community
Support
Support Questions
Getting Warning: Maximum heap size rounded up to 5...

Announcements

Welcome to the upgraded Community! Read this blog to see What’s New!

Options

Subscribe to RSS Feed
Mark Question as New
Mark Question as Read
Float this Question for Current User
Bookmark
Subscribe
Mute
Printer Friendly Page

Options

Subscribe to RSS Feed
Mark Question as New
Mark Question as Read
Float this Question for Current User
Bookmark
Subscribe
Mute
Printer Friendly Page

Getting Warning: Maximum heap size rounded up to 512 MB while running spark-shell

Labels:

Labels:
Apache Hadoop
Apache Hive
Apache Spark

KPG1

Explorer

Created ‎01-24-2023 09:56 AM

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content

01-24-2023 09:56:39

Even after passing the executor memory and driver memory as 1 Gb still it is getting maximum heap size rounded to 512 MB

spark-shell --conf spark.executor.memory=1g --conf spark.driver.memory=1g
Warning: Maximum heap size rounded up to 512 MB

Anything that is missing?

171 Views

0 Kudos

All forum topics
Previous
Next

0 REPLIES 0

Announcements

What's New @ Cloudera

Cloudera Streaming Analytics (CSA) 1.11 with support for Ice...

What's New @ Cloudera

Cloudera Operational Database (COD) supports creating an ope...

What's New @ Cloudera

Cloudera Operational Database (COD) is available as a Techni...

What's New @ Cloudera

Cloudera Operational Database (COD) has removed the COD_ON_G...

Community Announcements

Introducing the new and improved Community!

View More Announcements

Labels

Top Labels

Alphabetical

Apache NiFi 5,240
Apache Hive 4,609
Apache Hadoop 4,130
Apache Ambari 3,271
Apache Spark 2,505
Hortonworks Data Platform (HDP) 2,470
Cloudera Manager 2,279
Apache HBase 1,496
HDFS 1,492
Apache YARN 1,452
Apache Impala 1,137
Apache Kafka 951
Cloudera Enterprise Data Hub (CDH) 917
Apache Ranger 864
Apache Oozie 791
Apache Sqoop 772
Cloudera Hue 717
Kerberos 624
Cloudera Data Platform (CDP) 575
Security 484
Apache Atlas 463
Cloudera DataFlow (CDF) 426
Apache Zeppelin 418
Apache Pig 400
Apache Solr 396
Apache Zookeeper 345
Apache Phoenix 321
MapReduce 311
Apache Tez 289
Apache Flume 281
Hortonworks Cloudbreak 250
Apache Knox 235
Apache Kudu 214
Apache Storm 203
Cloudera Data Platform Private Cloud (CDP-Private) 187
Cloudera Data Science Workbench (CDSW) 174
NiFi Registry 170
Certification 164
Quickstart VM 158
Apache Sentry 153
Apache Falcon 149
Manual Installation 148
Apache Metron 145
Docker 129
Cloudera Navigator 79
Apache MiNiFi 74
Apache HCatalog 72
Apache Accumulo 68
Cloudera Navigator Encrypt 67
Training 65
Hortonworks SmartSense 60
Cloudera Search 59
Cloudera Data Engineering (CDE) 59
Schema Registry 54
Gateway 45
Ambari Blueprints 41
Cloudera Essentials 38
Cloudera Enterprise Data Hub 27
Cloudera Data Science and Engineering 27
Cloudera Machine Learning (CML) 21
Apache Flink 20
Data Analytics Studio 19
Apache Airflow 14
Cloudera Analytic DB 12
Cloudera Operational DB 9
Apache Calcite 8
Data Visualization 8
Cloudera Data Warehouse (CDW) 7
Cloudera Navigator Key Trustee 6
Hortonworks DataPlane 6
Hortonworks Stream Messaging Manager 5
Data Engineering UI 5
Cloudera Navigator Key Manager 3
Apache Ozone 3
Data Lifecycle Manager 2
Cloudera Streaming Analytics (CSA) 2
Cloudera Navigator Optimizer 2
IBM Db2 Big SQL 2
Cloudera Altus Director 1
Cloudera OnDemand 1
Tensorflow 1
Hortonworks Operational Services 1
Apache Crunch 1
Apache Iceberg 1

Previous
1 of 9
Next

Community Guidelines
Community FAQs

Terms & Conditions
Privacy Policy and Data Policy
Unsubscribe / Do Not Sell My Personal Information
Supported Browsers Policy

Apache Hadoop and associated open source project names are trademarks of the Apache Software Foundation. For a complete list of trademarks, click here.