Support Questions

Find answers, ask questions, and share your expertise

disable core dumps. spark\yarn

avatar
Rising Star

how i can disable all cores generated by applications? 
we are based on spark\yarn

6 REPLIES 6

avatar
Master Guru

@ilia987 In the service's configuration tab you should be able to see the Enable Core Dump property. You just have to uncheck and restart services if needed. 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Rising Star

Where exactly? i am using versions::

 

HDFS 3.1.1
YARN 3.1.1
MapReduce2 3.1.1
Tez 0.9.1
Hive 3.1.0
HBase 2.0.2
Pig 0.16.0
Sqoop 1.4.7
Oozie 4.3.1
ZooKeeper 3.4.6
Storm 1.2.1
Accumulo 1.7.0
Infra Solr 0.1.0
Ambari Metrics 0.1.0
Atlas 1.1.0
Kafka 2.0.0
Knox 1.0.0
Log Search 0.5.0
Ranger 1.2.0
Ranger KMS 1.2.0
SmartSense 1.5.1.2.7.3.0-139
Spark2 2.3.2
Zeppelin Notebook 0.8.0
Druid 0.12.1
Kerberos 1.10.3-30
Superset 0.23.0

 

avatar
Master Guru

@ilia987 Are you using Ambari to monitor the cluster? If yes you can refer this :

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-collect-debug-heap-dump-linux

 

If you are using Cloudera Manager then you can also goto CM > Service > Configuration tab. 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Rising Star

we are using ambari, but it is not what i wanted.
our hosts are ubuntu 18.04.
i tried the following:

  • set ulimit -c 0,
  • set disable core dump via code for our application:

 

#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
   
void spc_limit_core(void) {
  struct rlimit rlim;
   
  rlim.rlim_cur = rlim.rlim_max = 0;
  setrlimit(RLIMIT_CORE, &rlim);
}​

 

  • but when there is an error,( out of memory) an assert is generated and then a coredump,


     

avatar
Rising Star

anyone got an idea how i can disable core dumps?

avatar
Rising Star

bumping again,

 

still trying to disable  all application core dump that run inside the containers