Given below we have a number of subroutines and driver programs that show various methods of tracking memory usage from inside of a program. The file stuff.tgz contains all of the files.

The first set of files contains Blue Gene (Mc2) specific routines. The program domem.f90 does two things. It first allocates bigger and bigger blocks of memory and reports the heap and stack utilization as it progresses. This section of the core stops when the memory allocation routine fails. Next the program calls a recursive subroutine that allocates memory. The stack utilization is reported during the recursion and as the recursion unwinds.

domem.f90 example output

 IBM XL Fortran for Blue Gene, V14.1 (5799-AH1) Version 14.01.0000.0013
                               heap used    stack used    heap avail   stack avail
check   10            4096   290.816E+03    19.231E+03    17.079E+09    17.079E+09
check   11            8192   290.816E+03    19.231E+03    17.079E+09    17.079E+09
check   12           16384   290.816E+03    19.231E+03    17.079E+09    17.079E+09
check   13           32768   290.816E+03    19.231E+03    17.079E+09    17.079E+09
check   14           65536   290.816E+03    19.231E+03    17.079E+09    17.079E+09
check   15          131072   430.080E+03    19.231E+03    17.079E+09    17.079E+09
check   16          262144   430.080E+03    19.231E+03    17.079E+09    17.079E+09
check   17          524288   823.296E+03    19.231E+03    17.078E+09    17.078E+09
check   18         1048576     2.101E+06    19.231E+03    17.077E+09    17.077E+09
check   19         2097152     3.150E+06    19.231E+03    17.076E+09    17.076E+09
check   20         4194304     5.247E+06    19.231E+03    17.074E+09    17.074E+09
check   21         8388608     9.441E+06    19.231E+03    17.070E+09    17.070E+09
check   22        16777216    17.830E+06    19.231E+03    17.061E+09    17.061E+09
check   23        33554432    34.607E+06    19.231E+03    17.045E+09    17.045E+09
check   24        67108864    68.162E+06    19.231E+03    17.011E+09    17.011E+09
check   25       134217728   135.270E+06    19.231E+03    16.944E+09    16.944E+09
check   26       268435456   269.488E+06    19.231E+03    16.810E+09    16.810E+09
check   27       536870912   537.924E+06    19.231E+03    16.541E+09    16.541E+09
check   28      1073741824     1.075E+09    19.231E+03    16.004E+09    16.004E+09
check   29      2147483648     2.149E+09    19.231E+03    14.931E+09    14.931E+09
check   30      3221225472     3.222E+09    19.231E+03    13.857E+09    13.857E+09
check   31      4294967296     4.296E+09    19.231E+03    12.783E+09    12.783E+09
check   32      5368709120     5.370E+09    19.231E+03    11.709E+09    11.709E+09
check   33      6442450944     6.444E+09    19.231E+03    10.636E+09    10.636E+09
check   34      7516192768     7.517E+09    19.231E+03     9.562E+09     9.562E+09
check   35      8589934592     8.591E+09    19.231E+03     8.488E+09     8.488E+09
check   36      9663676416     9.665E+09    19.231E+03     7.414E+09     7.414E+09
check   37     10737418240    10.738E+09    19.231E+03     6.341E+09     6.341E+09
check   38     11811160064    11.812E+09    19.231E+03     5.267E+09     5.267E+09
check   39     12884901888    12.886E+09    19.231E+03     4.193E+09     4.193E+09
check   40     13958643712    13.960E+09    19.231E+03     3.119E+09     3.119E+09
check   41     15032385536    15.033E+09    19.231E+03     2.046E+09     2.046E+09
check   42     16106127360    16.107E+09    19.231E+03   972.007E+06   972.007E+06
 allocation error 17179869184
stack inside   59711  0
stack inside   99999  1
stack inside  140287  2
stack inside  180575  3
stack inside  220863  4
stack inside  261151  5
stack inside  301439  6
stack inside  341727  7
stack inside  382015  8
stack inside  422303  9
   unwinding  382015 10
   unwinding  341727 10
   unwinding  301439 10
   unwinding  261151 10
   unwinding  220863 10
   unwinding  180575 10
   unwinding  140287 10
   unwinding   99999 10
   unwinding   59711 10
 100000.0000
 

There are two programs in the linux diretory. The first, mallinfo.c, is from the page http://man7.org/linux/man-pages/man3/mallinfo.3.html This program calls the library routine mallinfo.

The mallinfo() function returns a copy of a structure containing
information about memory allocations performed by malloc(3) and
related functions.  This structure is defined as follows:

   struct mallinfo {
	   int arena;     /* Non-mmapped space allocated (bytes) */
	   int ordblks;   /* Number of free chunks */
	   int smblks;    /* Number of free fastbin blocks */
	   int hblks;     /* Number of mmapped regions */
	   int hblkhd;    /* Space allocated in mmapped regions (bytes) */
	   int usmblks;   /* Maximum total allocated space (bytes) */
	   int fsmblks;   /* Space in freed fastbin blocks (bytes) */
	   int uordblks;  /* Total allocated space (bytes) */
	   int fordblks;  /* Total free space (bytes) */
	   int keepcost;  /* Top-most, releasable space (bytes) */
   };

See: http://man7.org/linux/man-pages/man3/mallinfo.3.html for additional information.

The second program, minfo, is from http://man7.org/linux/man-pages/man3/malloc_info.3.html. It calls the routine malloc_info.

The malloc_info() function exports an XML string that describes the
current state of the memory-allocation implementation in the caller.
The string is printed on the file stream stream.  The exported string
includes information about all arenas (see malloc(3)).
See: http://man7.org/linux/man-pages/man3/malloc_info.3.html for additional information.

The final set of routines works by replacing the standard malloc and related calls with versions that track usage. There are three utility routines that report usage:

See the readme file for additional information.

There are two test programs in the test-malloc_count directory that show usage. The fortran program is allocates blocks of memory and reports usage. Next the program calls a recursive subroutine that allocates memory. The stack utilization is reported during the recursion and as the recursion unwinds.

The C program does a set of malloc()s and realloc()s and calls the routine memset() to allocate stack space.

ftest.f90 example output


                                            peak         current
                          before            8633            8633
                       allocated        40008633        40008633
                     deallocated        40008633            8633
                     reallocated        40008633           12633
stack before     268
stack inside    4200  1
stack inside   40272  2
stack inside   80320  3
stack inside  120368  4
stack inside  160416  5
stack inside  200464  6
stack inside  240512  7
stack inside  280560  8
stack inside  320608  9
   unwinding  280604 10
   unwinding  240556 10
   unwinding  200508 10
   unwinding  160460 10
   unwinding  120412 10
   unwinding   80364 10
   unwinding   40316 10
   unwinding     268 10
 stack after    4176
                      after call        40008633           12633
   90000.00    
malloc_count ### exiting, total: 40012633, peak: 40008633, current: 4000



test.c example output


malloc_count ### current 3145728, peak 3145728
malloc_count ### current 2097152, peak 3145728
malloc_count ### current 4194304, peak 4194304
malloc_count ### exiting, total: 12582912, peak: 4194304, current: 0
our peak memory allocation: 2097152
maximum stack usage: 66296



stuff.tgz 36956

(contains all of the files)



./bgq

Blue Gene (Mc2) only

domem.f90 2685

makefile 624

runit 1312

track.c 1172



./linux

Works on all platforms but output is obscure.

makefile 279

mallinfo.c 2081

minfo.c 2448

runit 574



./malloc_count-0.7.1

Replaces malloc with tracking version. Does not work on Mc2.

malloc_count.c 11069

malloc_count.h 2676

memprofile.h 5539

README.md 5741

stack_count.c 2928

stack_count.h 1982

./malloc_count-0.7.1/test-malloc_count

ftest.f90 1698

index.html 646

makefile 571

runit 556

test.c 2573

./malloc_count-0.7.1/test-memprofile

index.html 684

Makefile 372

memprofile.gnuplot 320

memprofile.pdf 20774

test.cc 1374