Friday, August 12, 2011

Memory Management - Preventing the kernel from dishing out more memory than required

I think for us who have been running computational jobs have seen the memory got eaten up by some buggy or stray applications. Hopefully the kernel kills it. But somehow you must have seen that the kernel may not have kill the culprit and the server go to a linbo.

Let's say if we wish to ensure that the kernel only gives out memory to processes equal to the physical memory, then we have to do the following at /etc/sysctl.conf or /etc/sysctl.d/myapp.conf

My assumption is that you have 10GB of swap and 20GB of memory and you wish the kernel to stop handling processes at  18GB RAM, then the calculation should be (swap size +  0.4 * RAM size)

So at /etc/sysctlf.conf, the configuration will be
vm.overcommit_memory = 2
vm.overcommit_ratio = 40
Note: The ratio is (40/100). For explanation of vm.overcommit_memory =2. Do look at Tweaking Linux Kernel Overcommit Behaviour for memory

Once the memory hits 18GB, the so-called OOM killer of the Linux kernel will kick in.

Another calculation example is that your RAM size and  SWAP size are the same and you wish exactly the physical memory to be used only. then
vm.overcommit_memory = 2
vm.overcommit_ratio = 0

For more information, do read
  1. When Linux runs out of memory 
  2. vm.overcommit_memory = 2, vm.overcommit_ratio = 0

No comments: