OutOfMemoryError: Kill Process or Sacrifice Child – Causes and Solutions

There are different flavors of OutOfMemoryError. One of the flavors of OutOfMemoryError is ‘Kill Process or sacrifice child’. This post discusses what triggers this ‘kill Process or sacrifice child’ OutOfMemoryError and potential solutions to diagnose this problem.

What Triggers OutOfMemory: Kill Process or Sacrifice Child?

When processes in the host tend to consume more memory than the available memory in the RAM, Operating System based on it’s internal heuristics computations will kill certain processes. In this circumstance, the processes which get killed by the Operating system will experience ‘OutOfMemoryError: Kill Process or sacrifice child.' This problem is more pervasive in the container’s world as well.

Simulating OutOfMemory: Kill Process or Sacrifice Child

In this section, let’s see how to simulate ‘Out Of memory: Kill Process or sacrifice child.’

We launched an AWS EC2 instance r5a.2xlarge Amazon Linux instance. This type of EC2 instance has 64GB RAM.

Fig 1: No Processes running in a r5a.2xlarge host that has 64GB RAM

In this EC2 instance, we launched one Java program. We configured this Java program to consume 20GB of RAM. So this means there is only an additional ~43.5GB of RAM is left. i.e.,

= 64 GB total RAM – 20GB Java Process – 0.5GB Kernel Processes

= 43.5GB

Fig 2: Launched a Java process that will occupy 20GB RAM

Now we went ahead and launched the BuggyApp. BuggyApp is a simple open-source chaos engineering program that is capable of generating various performance problems. We configured BuggyApp to keep creating objects. We set the initial heap size (i.e., Xms) of this program to be 1GB and maximum heap size (i.e., Xmx) to be 50GB and launched the BuggyApp. It means when BuggyApp starts, it’s initial memory consumption will be 1GB, and it has the potential to consume up to 50GB. But if you note, there is only room for 43.5GB in the RAM.

Fig 3: Launching BuggyApp with initial heap size to be 1GB and Max Heap size to be 50GB

BuggyApp got launched successfully, but when its memory consumption reached 43GB, the Operating system killed the BuggyApp program with the ‘Out Of memory: Kill Process or sacrifice child’ error. Operating system took this serious action so that other processes on the system can be safeguarded.

[ 1584.087068] Out of memory: Kill process 3070 (java) score 547 or sacrifice child 
[ 1584.094170] Killed process 3070 (java) total-vm:56994588kB, anon-rss:35690996kB, file-rss:0kB, shmem-
rss:0kB

Above is the kernel log message printed on this device. You can note that the Operating System is killing our BuggyApp java process (whose Process Id is 3070). Note Operating System has internal algorithms and heuristics and keeps track of score for each process. Score this BuggyApp got was 547, thus it killed this process.

Fig 4: BuggyApp experiencing Out Of memory: Kill Process or sacrifice child

Solutions for OutOfMemory: Kill Process or Sacrifice Child

Here are the potential solutions to fix ‘Out Of memory: Kill Process or sacrifice child’:

1. Increase the RAM capacity of the device in which your applications are running

2. Reduce the unnecessary processes that are running on the device.

3. Reduce the memory consumption of the processes which are consuming a lot of memory.