New Contributor
Posts: 2
Registered: ‎01-22-2016

swap alerts

I've configured all of my cloudera managed hosts to have vm.swappiness=1.  You can't turn off swap completely and have a stable machine, so that's as close as I'm going to get.


I still get constant alerts when a host pages 1500bytes to disk. The cloudera manager console goes yellow and my clients freak out.


What is up with the swap alert defaults alert level being set to such an asinine level?

Posts: 1,613
Kudos: 303
Solutions: 245
Registered: ‎07-31-2013

Re: swap alerts

The defaults were chosen after analysis of past swapping issues faced at various users. The moment swap starts getting hit, for a continuous workload system such as your cluster, the performance suffers very terribly and its worth alerting against.

In an ideal setup with the right heap configurations you shouldn't hit swap at all. Have you reviewed your host's role's heap usage and memory resource grants (if they run a NodeManager) to account for possible over-subscription that could cause the swap to kick in due to nearing RAM exhaustion?

While 1.5 KB isn't a huge number, you shouldn't be hitting it at all in the first place - worth trying to understand what the memory usage patterns were like when that happens.

Of course, if you are OK with a certain degree of swap on master hosts, you may tune the alerts. Its worse to see them on the slave hosts where the real user workloads run.
New Contributor
Posts: 1
Registered: ‎09-22-2017

Re: swap alerts

I have the same problem ~ ~
Physical memory only 3.1G / 11.8G,
KUDU Master warns that swap memory is being used
New Contributor
Posts: 2
Registered: ‎11-13-2017

Re: swap alerts



We have the same problem on both test and production cluster.
It was set up a few weeks ago, and nothing really demanding is running ( we have 100+GBs of free RAM on each node )
However, Hue, Sentry and Spark is showing warning for swappiness.

I also see nothing relevant in the logs.

Swappiness is 1 on each node.

Please, if you have any lead to go on let us know. Thank you :)

New Contributor
Posts: 2
Registered: ‎11-13-2017

Re: swap alerts

Okay, we found the reason for swapping.
It was not any of the Cloudera services installed, but some 3rd party app running in a docker container.

I told the admins that it is not the brightest idea to run apps on the management / worker nodes of a managed cluster, but that's jsut how things go here.

Hope this will eventually help someone too!

Take care folks!