Testing the cgroup memory is not something as easy as we can think. It can’t be
only question of malloc(100000) in a loop, as the Linux kernel overcommit
memory allocation: so even if we get effectively a 100000 bytes long memory
space, this doesn’t decrease the physical available memory. To do so, this
space need to be changed pages by pages, that can be tedious to do. And quite
uncertain, because the kernel can take advantage of the swap partition…
To facilitate memory testing, there is memhog, part of numactl tools. It
aims to allocate large memory block for testing, directly in the physical
memory.
If we ask memhog to allocate 100M, 1G, 5G, 10G, … it’ll actually ask the kernel for such amont of memory, in no time. If there is enough memory available, we got our 100M, 1G, 5G, … and the available memory is really decreased by such volume.
In my demos, I need a slightly different behaviour, to demonstrate what happens as a function of time.
I slightly modify memhog to introduce a timer between slice allocation.
/* Derivative work of memhog.c from the numactl repository,
hosted at https://github.com/numactl/numactl.git
Copyright (C) 2016 Pierre-Olivier Mercier, EPITA.
Copyright (C) 2003,2004 Andi Kleen, SuSE Labs.
numactl is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; version
2.
numactl is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should find a copy of v2 of the GNU General Public License somewhere
on your Linux system; if not, write to the Free Software Foundation,
Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
#include <stdio.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
char *
alloc_workbuf(size_t size)
{
char *ptr;
/* allocate some memory */
ptr = malloc(size);
/* return NULL on failure */
if (ptr == NULL)
{
fprintf(stderr, "Unable to malloc %zuk: %s\n", size / 1024, strerror(errno));
return NULL;
}
/* lock this buffer into RAM */
if (mlock(ptr, size)) {
free(ptr);
if (errno == ENOMEM)
fprintf(stderr, "Unable to lock memory: not enough permissions\n");
else
perror("Unable to lock memory");
return NULL;
}
return ptr;
}
int
main(int argc, char *argv[])
{
size_t size;
size_t steps = 32;
/* parse arguments */
if (argc < 2 || argc > 3) {
fprintf(stderr, "Usage: memhog <size in MB> [nb steps]\n");
return EXIT_FAILURE;
}
size = atoi(argv[1]);
if (argc > 2)
steps = atoi(argv[2]);
size = size / steps;
/* Do stuff */
for (size_t i = 0; i < steps; i++)
{
if (!alloc_workbuf(size * 1000000UL))
break;
printf(".");
fflush(stdout);
usleep(80000);
}
printf("\n");
return EXIT_SUCCESS;
}
You can compile those C lines with just make, stating your file is named memhog.c:
make memhog