Using malloc to Debug Memory Misuse in Cocoa

Every few months, there is a discussion on cocoa-dev or a question on stackoverflow.com that basically boils down to “I have a leak or over-release and I can’t use Instruments to debug it. Help?”.

Quite often, the questioner can actually use Instruments just fine, but simply lacks the know-how or hasn’t tried in a while and doesn’t realize that Instruments has improved significantly with each release of the developer tools. No, really, Instruments is a fantastic tool and I use it whenever I can; what you see below is for the exceptional case, not the norm.

There are cases where using Instruments is either inconvenient or impractical. Namely, trying to track down an intermittent crasher or trying to gain insight into memory leaks over a long running session will create a prohibitively large dataset for Instruments to process (Instruments allows for much more detailed analysis of the object graph and this analysis loads a lot more data than the tools I’ll demonstrate below).

Thus, it is helpful to be familiar with the rather powerful set of tools available from the command line and within the debugger.

Almost always, you are going to want to enable a bit of additional data via the malloc infrastructure. Have a look at the malloc(3) man page. There is an entire section devoted to ENVIRONMENT variables and there are a handful of extremely useful variables!

First and foremost, you are almost always going to want to use MallocStackLoggingNoCompact. When enabled, malloc stack logging writes the stack trace of every allocation and free event to an efficiently compact binary file in /tmp/ (it used to be in memory and, thus, used to be a great way to exhaust heap. No longer!!). Unfortunately, it doesn’t record the retain and release events, but simply knowing where the object was allocated is generally quite useful (it is generally relatively easy to track down who retained an object once you know which object it is). Under GC, you can set the AUTO_REFERENCE_COUNT_LOGGING and CFRetain/CFRelease events will be logged to the malloc history.

You can then use the malloc_history command line tool to query for all events related to a particular address in memory.

While malloc_history requires that the process still exists, it doesn’t have to be running! If you run your app under gdb, you can still use malloc_history to query the application even when it is stopped in the debugger!

Speaking of gdb, you can use the info malloc command in gdb to query the same information. Under GC, the info gc-roots and info gc-referers commands can be used to interrogate the collector for information about the connectivity of the object graph in your running application.

If you enable zombies via setting the NSZombieEnabled environment variable to YES, the address spewed in the error message when messaging a zombie can be passed directly to malloc_history.

The leaks command line tool scans memory and detects leaks in the form of allocations of memory for which the address to that memory is not stored anywhere else within the application. The leaks tool has been vastly improved in the Snow Leopard release of the Xcode tools; it is much much faster and spits out false positives almost never. It is still possible to have a leak that leaks cannot detect, of course. And, remember, even if you can still reach memory, it is still a total waste if you never use that memory’s contents again!

So, that is a brief summary of the state of command line memory debugging on Mac OS X as of Snow Leopard. Of course, that’s just a bunch of words. How about an example?

The code:

#import <Foundation/Foundation.h>
int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    BOOL doLeak = !!getenv("leak");
    id o = [NSObject new];
    o = [NSMutableSet new];
    if (doLeak) {
        // leak some memory
        [o retain];
        [o release];
        [o retain];
        [o autorelease];
        o = @"Uncle";
    } else {
        // message an overreleased object
        [o autorelease];
        [o release];
        o = @"Bob";
    }
    [pool drain];
    sleep(4242424242);
    return 0;
}

If you create a standard Cocoa Command Line Tool project and paste the above into the main file, you will have effectively created the tool I used for the below examples.

Detecting Leaks

To test for leaks, you’ll want to use the leaks command line tool, generally.

You can either set the environment variables in the shell and then run the command or do it all at once:

env MallocStackLoggingNoCompact=YES leak=YES /tmp/bbum-products/Debug/MemoryMunchies

Or, in gdb you can use the set env to set the environment variables prior to a run.

In any case, when you run the tool with the environment variables set as above, you’ll see spewage like this:

MemoryMunchies(56604) malloc: recording malloc stacks to disk using standard recorder
MemoryMunchies(56604) malloc: stack logging compaction turned off; size of log files on disk can increase rapidly
MemoryMunchies(56604) malloc: stack logs being written into /tmp/stack-logs.56604.MemoryMunchies.z05dcb.index

Now, you can use the leaks command to scan for leaks:

% leaks 56604
Process 56604: 845 nodes malloced for 204 KB
Process 56604: 2 leaks for 64 total leaked bytes.
Leak: 0x10010c8e0  size=48  zone: DefaultMallocZone_0x100004000	instance of 'NSCFSet', type ObjC, implemented in Foundation
	0x70d06df8 0x00007fff 0x00001080 0x00000001 	.m.p............
	0x00000001 0x00000000 0x00000000 0x00010000 	................
	0x70ef7268 0x00007fff 0x00000000 0x00000000 	hr.p............
	Call stack: [thread 0x7fff7087cbe0]: | start | main | -[__NSPlaceholderSet initWithCapacity:]
| CFSetCreateMutable | CFBasicHashCreate | _CFRuntimeCreateInstance
| malloc_zone_malloc
Leak: 0x100108ee0  size=16  zone: DefaultMallocZone_0x100004000	instance of 'NSObject', type ObjC, implemented in CoreFoundation
	0x70f124a8 0x00007fff 0x00000000 0x00000000 	.$.p............
	Call stack: [thread 0x7fff7087cbe0]: | start | main | +[NSObject(NSObject) new]
| +[NSObject(NSObject) allocWithZone:] | _internal_class_createInstanceFromZone
| calloc | malloc_zone_calloc

Note that there are actually two leaks here! I had inadvertently left in an extra object allocation as I hacked up that bit of code! Oops.

Or, if I had used Object Alloc in Instruments to grab the address of an interesting object or if I had an NSLog() somewhere that dumped the address of an interesting object (use %p to print pointers nicely), then I could use the malloc_history command to find more information:

% malloc_history 56604 0x10010c8e0
ALLOC 0x10010c8e0-0x10010c90f [size=48]: thread_7fff7087cbe0 |start | main
| -[__NSPlaceholderSet initWithCapacity:] | CFSetCreateMutable | CFBasicHashCreate
| _CFRuntimeCreateInstance | malloc_zone_malloc

In any case, both leaks and malloc_history make it abundantly clear that the allocation event occurred in the main() function. You will quickly learn to ignore the implementation details of the frameworks that these tools expose.

Note that it is quite helpful to enable scribbling via the Malloc environment variables. This will overwrite memory with a known set of non-pointer-esque values upon allocation and deallocation, thus preventing stale data from hiding leaks.

Debugging Crashes

Just like MallocStackLoggingNoCompact, zombies can be enabled via an environment variable; NSZombieEnabled. Or more specifically:

env MallocStackLoggingNoCompact=YES NSZombieEnabled=YES /tmp/bbum-products/Debug/MemoryMunchies

Of course, the app just crashes, leaving you without a process to interrogate. The log files in /tmp/ can’t be interrogated without the process still around.

So, we need to run it in gdb. Being lazy, I just do:

env MallocStackLoggingNoCompact=YES NSZombieEnabled=YES gdb /tmp/bbum-products/Debug/MemoryMunchies

There’ll be a bunch of malloc spewage as various sub-processes are launched and reaped. It can be safely ignored. Or, if you want, you could set up the instance variables in gdb:

% gdb /tmp/bbum-products/Debug/MemoryMunchies
(gdb) set env MallocStackLoggingNoCompact=YES
(gdb) set env NSZombieEnabled=YES
(gdb) r
Reading symbols for shared libraries .++++....................... done
MemoryMunchies(57523) malloc: recording malloc stacks to disk using standard recorder
MemoryMunchies(57523) malloc: stack logging compaction turned off; size of log files on disk can increase rapidly
MemoryMunchies(57523) malloc: stack logs being written into /tmp/stack-logs.57523.MemoryMunchies.E2tMo1.index
2010-01-10 16:49:47.564 MemoryMunchies[57523:903] *** -[CFSet release]: message sent to deallocated instance 0x10010d680
Program received signal SIGTRAP, Trace/breakpoint trap.
0x00007fff880ede36 in ___forwarding___ ()
(gdb)

At this point, the malloc_history command line tool will still work just fine. However, you can also interrogate malloc state from within gdb itself:

(gdb) info malloc 0x10010d680
Alloc: Block address: 0x000000010010d680 length: 48
Stack - pthread: 0x7fff7087cbe0 number of frames: 7
    0: 0x7fff84b4ea6e in malloc_zone_malloc
    1: 0x7fff8806e445 in _CFRuntimeCreateInstance
    2: 0x7fff8806e230 in CFBasicHashCreate
    3: 0x7fff8808af01 in CFSetCreateMutable
    4: 0x7fff880c809e in -[__NSPlaceholderSet initWithCapacity:]
    5: 0x100000e02 in main at /tmp/MemoryMunchies/MemoryMunchies.m:7
    6: 0x100000d70 in start

Note that if there are multiple allocation events at that address, the events will be printed in chronological order. That is, oldest — and most interesting — events will be last.

This entry was posted on Sunday, January 10th, 2010 at 8:08 pm and is filed under Cocoa, Mac OS X, Objective-C. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.