Using malloc to Debug Memory Misuse in Cocoa

Every few months, there is a discussion on cocoa-dev or a question on stackoverflow.com that basically boils down to “I have a leak or over-release and I can’t use Instruments to debug it. Help?”.

Quite often, the questioner can actually use Instruments just fine, but simply lacks the know-how or hasn’t tried in a while and doesn’t realize that Instruments has improved significantly with each release of the developer tools. No, really, Instruments is a fantastic tool and I use it whenever I can; what you see below is for the exceptional case, not the norm.

There are cases where using Instruments is either inconvenient or impractical. Namely, trying to track down an intermittent crasher or trying to gain insight into memory leaks over a long running session will create a prohibitively large dataset for Instruments to process (Instruments allows for much more detailed analysis of the object graph and this analysis loads a lot more data than the tools I’ll demonstrate below).

Thus, it is helpful to be familiar with the rather powerful set of tools available from the command line and within the debugger.

Almost always, you are going to want to enable a bit of additional data via the malloc infrastructure. Have a look at the malloc(3) man page. There is an entire section devoted to ENVIRONMENT variables and there are a handful of extremely useful variables!

First and foremost, you are almost always going to want to use MallocStackLoggingNoCompact. When enabled, malloc stack logging writes the stack trace of every allocation and free event to an efficiently compact binary file in /tmp/ (it used to be in memory and, thus, used to be a great way to exhaust heap. No longer!!). Unfortunately, it doesn’t record the retain and release events, but simply knowing where the object was allocated is generally quite useful (it is generally relatively easy to track down who retained an object once you know which object it is). Under GC, you can set the AUTO_REFERENCE_COUNT_LOGGING and CFRetain/CFRelease events will be logged to the malloc history.

You can then use the malloc_history command line tool to query for all events related to a particular address in memory.

While malloc_history requires that the process still exists, it doesn’t have to be running! If you run your app under gdb, you can still use malloc_history to query the application even when it is stopped in the debugger!

Speaking of gdb, you can use the info malloc command in gdb to query the same information. Under GC, the info gc-roots and info gc-referers commands can be used to interrogate the collector for information about the connectivity of the object graph in your running application.

If you enable zombies via setting the NSZombieEnabled environment variable to YES, the address spewed in the error message when messaging a zombie can be passed directly to malloc_history.

The leaks command line tool scans memory and detects leaks in the form of allocations of memory for which the address to that memory is not stored anywhere else within the application. The leaks tool has been vastly improved in the Snow Leopard release of the Xcode tools; it is much much faster and spits out false positives almost never. It is still possible to have a leak that leaks cannot detect, of course. And, remember, even if you can still reach memory, it is still a total waste if you never use that memory’s contents again!

So, that is a brief summary of the state of command line memory debugging on Mac OS X as of Snow Leopard. Of course, that’s just a bunch of words. How about an example?

The code:

#import <Foundation/Foundation.h>
int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    BOOL doLeak = !!getenv("leak");
    id o = [NSObject new];
    o = [NSMutableSet new];
    if (doLeak) {
        // leak some memory
        [o retain];
        [o release];
        [o retain];
        [o autorelease];
        o = @"Uncle";
    } else {
        // message an overreleased object
        [o autorelease];
        [o release];
        o = @"Bob";
    }
    [pool drain];
    sleep(4242424242);
    return 0;
}

If you create a standard Cocoa Command Line Tool project and paste the above into the main file, you will have effectively created the tool I used for the below examples.

Detecting Leaks

To test for leaks, you’ll want to use the leaks command line tool, generally.

You can either set the environment variables in the shell and then run the command or do it all at once:

env MallocStackLoggingNoCompact=YES leak=YES /tmp/bbum-products/Debug/MemoryMunchies

Or, in gdb you can use the set env to set the environment variables prior to a run.

In any case, when you run the tool with the environment variables set as above, you’ll see spewage like this:

MemoryMunchies(56604) malloc: recording malloc stacks to disk using standard recorder
MemoryMunchies(56604) malloc: stack logging compaction turned off; size of log files on disk can increase rapidly
MemoryMunchies(56604) malloc: stack logs being written into /tmp/stack-logs.56604.MemoryMunchies.z05dcb.index

Now, you can use the leaks command to scan for leaks:

% leaks 56604
Process 56604: 845 nodes malloced for 204 KB
Process 56604: 2 leaks for 64 total leaked bytes.
Leak: 0x10010c8e0  size=48  zone: DefaultMallocZone_0x100004000	instance of 'NSCFSet', type ObjC, implemented in Foundation	
	0x70d06df8 0x00007fff 0x00001080 0x00000001 	.m.p............
	0x00000001 0x00000000 0x00000000 0x00010000 	................
	0x70ef7268 0x00007fff 0x00000000 0x00000000 	hr.p............
	Call stack: [thread 0x7fff7087cbe0]: | start | main | -[__NSPlaceholderSet initWithCapacity:] 
| CFSetCreateMutable | CFBasicHashCreate | _CFRuntimeCreateInstance 
| malloc_zone_malloc 
Leak: 0x100108ee0  size=16  zone: DefaultMallocZone_0x100004000	instance of 'NSObject', type ObjC, implemented in CoreFoundation	
	0x70f124a8 0x00007fff 0x00000000 0x00000000 	.$.p............
	Call stack: [thread 0x7fff7087cbe0]: | start | main | +[NSObject(NSObject) new] 
| +[NSObject(NSObject) allocWithZone:] | _internal_class_createInstanceFromZone 
| calloc | malloc_zone_calloc 

Note that there are actually two leaks here! I had inadvertently left in an extra object allocation as I hacked up that bit of code! Oops.

Or, if I had used Object Alloc in Instruments to grab the address of an interesting object or if I had an NSLog() somewhere that dumped the address of an interesting object (use %p to print pointers nicely), then I could use the malloc_history command to find more information:

% malloc_history 56604 0x10010c8e0
ALLOC 0x10010c8e0-0x10010c90f [size=48]: thread_7fff7087cbe0 |start | main 
| -[__NSPlaceholderSet initWithCapacity:] | CFSetCreateMutable | CFBasicHashCreate 
| _CFRuntimeCreateInstance | malloc_zone_malloc 

In any case, both leaks and malloc_history make it abundantly clear that the allocation event occurred in the main() function. You will quickly learn to ignore the implementation details of the frameworks that these tools expose.

Note that it is quite helpful to enable scribbling via the Malloc environment variables. This will overwrite memory with a known set of non-pointer-esque values upon allocation and deallocation, thus preventing stale data from hiding leaks.

Debugging Crashes

Just like MallocStackLoggingNoCompact, zombies can be enabled via an environment variable; NSZombieEnabled. Or more specifically:

env MallocStackLoggingNoCompact=YES NSZombieEnabled=YES /tmp/bbum-products/Debug/MemoryMunchies

Of course, the app just crashes, leaving you without a process to interrogate. The log files in /tmp/ can’t be interrogated without the process still around.

So, we need to run it in gdb. Being lazy, I just do:

env MallocStackLoggingNoCompact=YES NSZombieEnabled=YES gdb /tmp/bbum-products/Debug/MemoryMunchies

There’ll be a bunch of malloc spewage as various sub-processes are launched and reaped. It can be safely ignored. Or, if you want, you could set up the instance variables in gdb:

% gdb /tmp/bbum-products/Debug/MemoryMunchies
(gdb) set env MallocStackLoggingNoCompact=YES
(gdb) set env NSZombieEnabled=YES
(gdb) r
Reading symbols for shared libraries .++++....................... done
MemoryMunchies(57523) malloc: recording malloc stacks to disk using standard recorder
MemoryMunchies(57523) malloc: stack logging compaction turned off; size of log files on disk can increase rapidly
MemoryMunchies(57523) malloc: stack logs being written into /tmp/stack-logs.57523.MemoryMunchies.E2tMo1.index
2010-01-10 16:49:47.564 MemoryMunchies[57523:903] *** -[CFSet release]: message sent to deallocated instance 0x10010d680
Program received signal SIGTRAP, Trace/breakpoint trap.
0x00007fff880ede36 in ___forwarding___ ()
(gdb)

At this point, the malloc_history command line tool will still work just fine. However, you can also interrogate malloc state from within gdb itself:

(gdb) info malloc 0x10010d680
Alloc: Block address: 0x000000010010d680 length: 48
Stack - pthread: 0x7fff7087cbe0 number of frames: 7
    0: 0x7fff84b4ea6e in malloc_zone_malloc
    1: 0x7fff8806e445 in _CFRuntimeCreateInstance
    2: 0x7fff8806e230 in CFBasicHashCreate
    3: 0x7fff8808af01 in CFSetCreateMutable
    4: 0x7fff880c809e in -[__NSPlaceholderSet initWithCapacity:]
    5: 0x100000e02 in main at /tmp/MemoryMunchies/MemoryMunchies.m:7
    6: 0x100000d70 in start

Note that if there are multiple allocation events at that address, the events will be printed in chronological order. That is, oldest — and most interesting — events will be last.



4 Responses to “Using malloc to Debug Memory Misuse in Cocoa”

  1. M. Noel says:

    Thanks for the very helpful information. In particular, zombies have been a lifesaver for me, especially in the past year.

    Whenever I’ve used the leaks tool in the past, legitimate leaks in my own code have been buried beneath objects and strings in Apple frameworks (which I assume are false positives and not actual leaks). It sounds like the tool’s been beefed up, and I’ll have to give it another shot.

    I love the retain/release history for zombies in Instruments in 10.6, but it’d be nice to have a way to log retain/release information for my objects during my automated tests. In fact, I’d just settle for a deterministic way to automatically detect Cocoa leaks. Overriding operator new in C++ and using my own CFAllocator for CFTypeRefs has been helpful, but I haven’t been able to come up with a surefire way to get that information for Cocoa objects the framework allocates on my behalf (like copied strings). Swizzling alloc/dealloc/retain/release/copy doesn’t seem like it would work out very well, especially for class clusters, since unlike C++ operator new they’re not actually guaranteed to allocate or deallocate, or even change a ref count. Any ideas? For example, is NSRecordAllocationEvent (from NSDebug.h) useful to third parties, and if so, are there any examples for how we’d use it? Anyhow, if you have any suggestions for how we should automate detecting Cocoa leaks, I’d love to hear it.

  2. Sean M says:

    Bill, perhaps you’d have a definitive answer to this: is MallocPreScribble real? This page http://developer.apple.com/mac/library/releasenotes/DeveloperTools/RN-MallocOptions/index.html lists it as a newer feature, but the malloc man page does not mention it. IIRC, it was discussed on an apple list and the conclusion was that MallocPreScribble is bogus and that MallocScribble scribbles with new allocations and when deallocated.

  3. malloc Debugging : Light Year Blog says:

    [...] Debugging Tuesday, January 12, 2010 Bill Bumgarner posted a nice tutorial on using malloc to debug memory misuse in Cocoa. I’ve run across these before when reading the malloc(1) man page, but it’s nice to [...]

  4. harry.ee says:

    Hi Bill, thanks for your blog.
    It helped me find incompatible designs between Garbage Collector Apps and NSInvocation (while using DO).
    I have random crashes when I call methods which return objects by reference.
    If you have any work around, any help will be appreciated.

Leave a Reply

Line and paragraph breaks automatic.
XHTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>