Support Questions

Find answers, ask questions, and share your expertise
Announcements
Welcome to the upgraded Community! Read this blog to see What’s New!

disable core dumps. spark\yarn

avatar
Explorer

how i can disable all cores generated by applications? 
we are based on spark\yarn

6 REPLIES 6

avatar

@ilia987 In the service's configuration tab you should be able to see the Enable Core Dump property. You just have to uncheck and restart services if needed. 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

Where exactly? i am using versions::

 

HDFS 3.1.1
YARN 3.1.1
MapReduce2 3.1.1
Tez 0.9.1
Hive 3.1.0
HBase 2.0.2
Pig 0.16.0
Sqoop 1.4.7
Oozie 4.3.1
ZooKeeper 3.4.6
Storm 1.2.1
Accumulo 1.7.0
Infra Solr 0.1.0
Ambari Metrics 0.1.0
Atlas 1.1.0
Kafka 2.0.0
Knox 1.0.0
Log Search 0.5.0
Ranger 1.2.0
Ranger KMS 1.2.0
SmartSense 1.5.1.2.7.3.0-139
Spark2 2.3.2
Zeppelin Notebook 0.8.0
Druid 0.12.1
Kerberos 1.10.3-30
Superset 0.23.0

 

avatar

@ilia987 Are you using Ambari to monitor the cluster? If yes you can refer this :

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-collect-debug-heap-dump-linux

 

If you are using Cloudera Manager then you can also goto CM > Service > Configuration tab. 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

we are using ambari, but it is not what i wanted.
our hosts are ubuntu 18.04.
i tried the following:

  • set ulimit -c 0,
  • set disable core dump via code for our application:

 

#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
   
void spc_limit_core(void) {
  struct rlimit rlim;
   
  rlim.rlim_cur = rlim.rlim_max = 0;
  setrlimit(RLIMIT_CORE, &rlim);
}​

 

  • but when there is an error,( out of memory) an assert is generated and then a coredump,


     

avatar
Explorer

anyone got an idea how i can disable core dumps?

avatar
Explorer

bumping again,

 

still trying to disable  all application core dump that run inside the containers

Labels