Latest change on $Date: 2009/07/06 12:30:17 $.

Malloc Debug Library for C on Unix

by Rammi

Version 1.21

Content

What for?

When writing C programs many of the bugs are memory related. They may result in strange behaviour which can arouse far away from the erroneous position.

I once spent about thirty hours hunting for an annoying bug. The setting was bad. In a big project where a program (for which we don't had the source) read in libraries (written by us) at runtime and then used them a segmentation fault occured my part when allocating memory. I had just written some arcane stuff concerning binary file i/o and wasn't surprised that much. Of course it was not possible to use a debugger and we had to switch to good old printf and visual trace (step the code in wetware memory line by line) debugging. But I did not find nothing, everything seemed wonderful -- except that the program still kept crashing.

It was a weekend and I hadn't had Internet access that time (it's seems so long ago that I remember dinosaurs stamping by my window). So I wrote the first version of the malloc debug library to see what happened. I linked it to my library and -- did not find anything. So at last (it was a weekend) I linked it to the libraries of my coworkers and indeed there was a off-by-one error in someone else's library. We love C, don't we?

Since then I always use it in my projects from the very beginning and do only switch it off for release versions. It's an unrenouncable improvement to find memory bugs as early as possible without doing any more extra work than including a header and setting a command line switch in the Makefile.

How it works

It's a simple thing. The malloc debug library implements wrappers for the normal heap handling functions:

When allocating memory the wrapper functions demand some more memory from the system and use this extra space to write some special bytefields before and after the buffer they finally return to the user. Some of the extra space before the buffer is used for extra information e.g. about the file position where the allocation took place. When freeing the buffer (and maybe more often) the bytefields are controlled whether they are unchanged. If there is a changed the program is aborted immediately with an error message indicating the position where the erroneous buffer was allocated and where the error was detected. E.g. a common error:

<MALLOC_DEBUG>  Corrupted block end (possibly written past the end)
        should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
        is:        00a5a5a5 5b5b5b5b abababab aa55aa55
        block was allocated in rtest.c:181 [4 Bytes, generation 1]
        error was detected in  rtest.c:185
        Looks like string allocated one byte too short
                (missing the closing zero)
      

The abort allows you to switch on your debugger and inspect things more closely.

Getting it

The archive rmdebug.tgz is a gzipped tar archive, has about 15k and contains the files:

Get rmalloc.tgz over HTTP

REMINDER:
You may download/use/change/redistribute this utility as you like but you do this at your own risk without any warranty.

Installation Guide

Get rmalloc.c and rmalloc.h from downloading rmalloc.tgz from above. Compile rmalloc.c to rmalloc.o and include rmalloc.h in each of your files after the standard includes. Recompile your stuff with MALLOC_DEBUG defined and link rmalloc.o to it. That's all to use it.

Detailed Information

First follow the Installation Guide.

Switching it on/off

If your sources are compiled with MALLOC_DEBUG the calls to the standard malloc library are exchanged by the debug malloc library functions. If you don't define MALLOC_DEBUG the calls stay as they are. Be sure that you have included rmalloc.h everywhere! Also be sure that you recompile everything after adding or removing MALLOC_DEBUG in your Makefile.

Testing it

The package includes a Makefile and a file rtest.c. Just say make and you can run rtest which should give you some output similar to this:

------------------
Running test  0...
------------------
<MALLOC_DEBUG>	rmalloc -- malloc wrapper V 1.21
	by Rammi <mailto:rammi@hexco.de>
	Compiled with following options:
		testing:	only actual block
		generations:	ON
		eloquence:	OFF
		realloc(0):	NOT ALLOWED
		free(0):	NOT ALLOWED
		flags:	USED
		alignment:	8
		pre space:	32
		post space:	16
		hash tab size:	257

<MALLOC_STATS>	============ STATISTICS (rtest.c:90) =============
<MALLOC_STATS>	  1000 x        8 Bytes in rtest.c:84, generations: 1 2 3 ...
<MALLOC_STATS>	*Variable*	        8000 Bytes
<MALLOC_STATS>	*Static*	           0 Bytes
<MALLOC_STATS>	*Total*	        8000 Bytes
<MALLOC_STATS>	============ END OF STATISTICS =============
<MALLOC_STATS>	============ STATISTICS (rtest.c:96) =============
<MALLOC_STATS>	  1000 x        8 Bytes in rtest.c:84, generations: 1 2 3 ...
<MALLOC_STATS>	    20 x        8 Bytes in rtest.c:93, generations: 1001 1002 1003 ...
<MALLOC_STATS>	*Variable*	        8160 Bytes
<MALLOC_STATS>	*Static*	           0 Bytes
<MALLOC_STATS>	*Total*	        8160 Bytes
<MALLOC_STATS>	============ END OF STATISTICS =============
<MALLOC_STATS>	============ STATISTICS (rtest.c:105) =============
<MALLOC_STATS>	    20 x        8 Bytes in rtest.c:93, generations: 1001 1002 1003 ...
<MALLOC_STATS>	  1000 x       16 Bytes in rtest.c:100, generations: 1021 1022 1023 ...
<MALLOC_STATS>	*Variable*	       16160 Bytes
<MALLOC_STATS>	*Static*	           0 Bytes
<MALLOC_STATS>	*Total*	       16160 Bytes
<MALLOC_STATS>	============ END OF STATISTICS =============
<MALLOC_DEBUG>	Corrupted block end (possibly written past the end)
	should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
	is:        a5a5a53f 5b5b5b5b abababab aa55aa55
	block was allocated in rtest.c:100 [16 Bytes, generation 2020]
	error was detected in rtest.c:119


------------------
Running test  1...
------------------
<MALLOC_DEBUG>	Corrupted block end (possibly written past the end)
	should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
	is:        00a5a5a5 5b5b5b5b abababab aa55aa55
	block was allocated in rtest.c:144 [3 Bytes, generation 1]
	error was detected in rtest.c:146
	Looks like string allocated one byte too short
		(forgetting the nul byte)


------------------
Running test  2...
------------------
<MALLOC_DEBUG>	Corrupted block end (possibly written past the end)
	should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
	is:        326c6f6e 67005b5b abababab aa55aa55
	block was allocated in rtest.c:163 [2 Bytes, generation 1]
	error was detected in rtest.c:165
	Looks somewhat like a too long string,
		ending with "2long"


------------------
Running test  3...
------------------
<MALLOC_DEBUG>	Corrupted block end (possibly written past the end)
	should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
	is:        e8530508 5b5b5b5b abababab aa55aa55
	block was allocated in rtest.c:183 [4 Bytes, generation 1]
	error was detected in rtest.c:187
	First 4 bytes of overwritten memory can be interpreted
		as a pointer to a block  allocated in:
		rtest.c:184 [8 Bytes, generation 2]


------------------
Running test  4...
------------------
<MALLOC_DEBUG>	Double or false delete
	Heap adress of block: 0x80560f0
	Detected in rtest.c:242
	Trying identification (may be incorrect!):
		Allocated in rtest.c:227 [2 Bytes]


------------------
Running test  5...
------------------
<MALLOC_DEBUG>	Double or false delete
	Heap adress of block: 0x12345678
	Detected in rtest.c:260


------------------
Running test  6...
------------------
<MALLOC_DEBUG>	WARNING: calloc() overflow! Returning NULL (in rtest.c:276)
<MALLOC_DEBUG>	Trying to free NULL pointer (in rtest.c:277)


------------------------------
All tests passed successfully.
------------------------------
      

Changing the behaviour

You can tweak the behaviour of the library in two ways:

  1. Change some switches in rmalloc.c
  2. Add some macros to your code

Switches in rmalloc.c

You can change the overall behaviour by setting different switches in rmalloc.c. Doing this has the advantage that you only have to recompile and relink rmalloc.c without touching anything else.

Additional macros

To use this macros you have to make changes to your code. I agree that this is ugly but you will get something for it. All these macros expand to nothing when MALLOC_DEBUG is undefined.

Reliability

This library was used on several platforms (at least Irix, Solaris, HPUX, Digital Unix, and Linux) by different users. It can help you find common errors like (moderate) overwriting of memory bounds or memory leaks. No problem using it with debuggers or other tools with different implementations of the standard malloc library because it internally uses it.

It is by no means perfect. Writing to dangling pointers may or may not be found. Fat overwrite (more than 16 bytes) may cause a crash of the library itself. Mixing the library with calls to the standard malloc library (because you don't include rmalloc.h everywhere or have compiled parts of your program with MALLOC_DEBUG defined and others not) will cause problems (e.g. when you use standard free with rmalloc'ed stuff and vice versa).

This is especially true when you use other library functions returning memory from heap than the wrapped ones. You will get a Double or false delete error when you free that memory. It is simple to write a wrapper yourself for that functons. Just look how I handled the getcwd library function.
Attention! The code was broken in version 1.13 which was fixed in 1.14.

Despite of the warnings above I use it always (except for releases) and haven't had these problems. But of course I know what I am doing.

Other malloc debug libraries

WARNING: Please note that I wrote this section in 1996 and have never updated it.

There are lots of other libraries which do a similar thing (it's a common problem). If you have already used another library and you are content with it, there is possibly no need to switch. If this is your first encounter with something like this you should give it a try.

I haven't had much need to use other malloc debug libraries, of course. I used a very early release of Purify and I have tested electric fence some time ago.

Purify

I liked Purify a lot but it is incredible expensive and makes your program slow because every memory access is monitored. So it is probably only used if you really know that you have problems. So if the problems don't show during test phase it's not used. But as we all know the problem will show when we present our program to the customer.

If your boss listens to you tell him to buy Purify anyway (it's not your money and you will be glad to have it when you need it). But until then you should use something else.

Electric Fence

Electric fence is freeware, too, but it uses a different concept compared to my stuff. It is programming your machine's MMU to protect the space behind the allocated blocks. If you access that space you will immediately get a segmentation fault. This is neat because you are really lead to the place where you actually write over the bounds. It will even react on reading beyond bounds. And it is fast because it uses hardware. But it uses more heap space (which tends to make it slower on big programs) and it doesn't find all errors when writing over the bounds because of alignment problems. Especially the problem of Test 1 of rtest is not found.

Releases

Some info on the various releases. Check here if you use older versions to see whether it is useful to update (of course it nearly always should). The most recent version appears at the top. Not all versions are described here because I was too lazy to start this section from the beginning.

License

Because I was asked to clarify the license of rmalloc from October 8 2010 it is coming under the ISC license (the main difference to the relaxed state before is that you'll have to keep the license text after modifications, which before was not required; otherwise you are free to do what you like):

Copyright © 2010  Andreas M. Rammelt <rammi@hexco.de>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
      

Further Information

If you need further information feel free to ask me.