+initialize Can Be Executed Multiple Times (+load not so much)

Some confusion on StackOverflow led to a massive string of comments. This is a question that comes up often, so here is some google fodder.

In Objective-C, a class can implement +initialize. This method will be invoked the first time the class is touched, prior to any other methods (other than +load).

The documentation says:

The runtime sends initialize to each class in a program exactly one time just before the class, or any class that inherits from it, is sent its first message from within the program.

Which is exactly true. But your +initialize methods can still be executed more than once!

Specifically, if a subclass does not implement +initialize but its superclass does, then that superclass’s +initialize will be invoked once per non-implementing subclass and once for itself.

An example (Foundation Tool, Garbage Collected):

@interface Abstract: NSObject
@end
@implementation Abstract
+ (void) initialize
{
    NSLog(@"Initializing %@", self);
}

+ (void) load
{
    NSLog(@"Loading");
}
@end

@interface Sub : Abstract
@end
@implementation Sub
@end

int main (int argc, const char * argv[]) {
    [Sub class];
    return 0;
}

This will output:

ArgyBargy[3720:903] Loading
ArgyBargy[3720:903] Initializing Abstract
ArgyBargy[3720:903] Initializing Sub

Which is why most +initialize methods are implemented as:

@implementation MyClass
+ (void) initialize
{
    if (self == [MyClass class]) {
        // ... do +init stuff here ...
    }
}
...
@end

Now, categories can seriously screw things up (as usual). Namely, if you implement +initialize in a category, it will override the classes +initialize. However, a category provided +load will not; both the category’s and the class’s +load methods will be invoked.

If you were to add the following category to the Sub/Abstract/NSObject example above:

@interface Abstract(Cat)
@end
@implementation Abstract(Cat)
+ (void) load
{
    NSLog(@"Category +load");
}
+ (void) initialize
{
    NSLog(@"Category +initialize %@", self);
}
@end

The program will spew:

ArgyBargy[3919:903] Loading
ArgyBargy[3919:903] Category +load
ArgyBargy[3919:903] Category +initialize Abstract
ArgyBargy[3919:903] Category +initialize Sub

Keep in mind, as well, that the runtime sends +initialize “in a thread-safe manner”. That implies that there is a lock involved somewhere within which then also implies that you better not block on a lock in your +initialize because whoever is supposed to unlock the lock might end up blocking on +initializes lock.

Or, to put it more bluntly, do not do any heavy lifting in +initialize. Keep it super simple & fast.

For me, +initialize is to be used only as a method of last resort. Well, 2nd to last. Last resort is a constructor attributed function (or +load).In the comments, James says:

+initialize is a great place to do heavy lifting. … Your objection about not performing any locks in +initialize is non-sensical. The runtime won’t allow any other other threads to send a message to the class until the +initialize message returns.

That claim is wrong.

While it is a great place to, say, initialize very simple global state– string constants, the date, etc… — +initialize is a horrible place to do any kind of heavy lifting for several reasons.

Most critically, it is impossible to know what order the classes will be +initialize‘d and said order will change across system configurations and, even, software updates. This leads to a need to do some kind of exclusion locking or something such that the initialization code paths are only followed once. But, more often than not, “heavy lifting” in the +initialize will trigger other classes to be initialized.

End result? A mess of interleaved locks whose complexity quickly grows out of control.

Here is a dead simple example of a +initialize deadlock:

#import <Foundation/Foundation.h>

@interface Bleep : NSObject
@end
@interface Blorp : NSObject
@end

static NSLock *initializationLock;

@implementation Bleep
+ (void) initialize
{
    NSLog(@"%s", class_getName(self));
    [initializationLock lock];
    NSLog(@"%@", [Blorp new]);
    [initializationLock unlock];
}
@end

@implementation Blorp
+ (void) initialize
{
    NSLog(@"%s", class_getName(self));
    [initializationLock lock];
    NSLog(@"%@", [Bleep new]);
    [initializationLock unlock];
}
@end

int main (int argc, const char * argv[]) {
    initializationLock = [NSLock new];
    [Blorp new];
    return 0;
}

Conveniently, the runtime detects the deadlock for us:

209 BleepBlorb[33953:903] Blorp
213 BleepBlorb[33953:903] Bleep
214 BleepBlorb[33953:903] *** -[NSLock lock]: deadlock (<NSLock: 0x20000f340> '(null)')
214 BleepBlorb[33953:903] *** Break on _NSLockError() to debug.

And the backtrace at the point of deadlock is interesting beyond the obvious failure:

#0  semaphore_wait_signal_trap ()
#1  pthread_mutex_lock ()
#2  -[NSLock lock] ()
#3  +[Bleep initialize] (self=0x100001120, _cmd=0x7fff8790a148) at /tmp/BleepBlorb/BleepBlorb.m:15
#4  _class_initialize ()
#5  prepareForMethodLookup ()
#6  lookUpMethod ()
#7  objc_msgSend ()
#8  +[Blorp initialize] (self=0x1000010d0, _cmd=0x7fff8790a148) at /tmp/BleepBlorb/BleepBlorb.m:26
#9  _class_initialize ()
#10 prepareForMethodLookup ()
#11 lookUpMethod ()
#12 objc_msgSend ()
#13 main (argc=1, argv=0x7fff5fbff7e0) at /tmp/BleepBlorb/BleepBlorb.m:37

In particular, it shows that the runtime, in and of itself, can trigger class initialization as a part of method lookup, thus implying even less control over initialization order than one might assume.

Now, of course, this is a contrived example. However, this particular pattern of dysfunctionality is quite easy to grow into as the complexity of initialization increases. Worse, it can be equally as easy to end up in a situation where the deadlock only occurs some of the time; only on some app launches on some subset of customers machines.

This isn’t just conjecture. I have intermittently spent quite a few hours at a time debugging this exact kind of deadlock across various applications (that shall remain nameless– popular apps, they were).

For the record, +load is an even worse time to do heavy lifting. The runtime environment is in an extremely non-deterministic state at that time.



15 Responses to “+initialize Can Be Executed Multiple Times (+load not so much)”

  1. James says:

    +initialize is actually a great place to do heavy lifting, because it isn’t invoked until the class is referenced making it appropriate point to implement lazy initialization of class variables. Implementing intensive code in +load just delays application startup, by classes that might be rarely (or never) used.

    Your objection about not performing any locks in +initialize is non-sensical. The runtime won’t allow any other other threads to send a message to the class until the +initialize message returns. This doesn’t/shouldn’t impact any other well-behaved thread synchronization within the current thread. This just make it more useful for implementing time-consuming lazy initialization, not less.

  2. bbum says:

    Oh, how wrong you are! The problem isn’t the class itself, it is what other classes you might touch. And if any of those classes touch the first class, oops.

    I speak from experience having had to track down and debug literally dozens of deadlocks in +initialize over the past decade.

  3. iWyre says:

    […] Bumgarner on +initialize by admin on September 9th, 2009 This is a must read for anyone who overrides +initialize. I’ll be honest, I either didn’t know about this […]

  4. James says:

    Oh, how wrong you are! ;)

    You may have caused deadlock issues using non-reentrant locks or through some other path, but the +initialize method is wrapped by a recursive lock that will not deadlock through circular references. Thus, class One’s +initialize can invoke class Two’s +initialize, which can safely use class One without any problems. Sample code would be easy to produce.

    If you’ve encountered deadlocks in your +initialization code, then it’s due to a flaw in your thread synchronization design. It is not inherently a flaw in how +initialize is designed or implemented.

    Your suggestion of using +load is far more dangerous and undesirable. Running code at +load, you may interact with classes that are not yet initialized. And worse, you push initialization code to application load time which simply slows launch times, rather that pushing time consuming initialization to +initialization time (or later) where it produces more efficient and responsive applications.

    I’ve used +initialize effectively for many years and have never had a problem with it. I continue to recommend +initialize in all my books as the best solution to lazily initialize static/global class data structures.

  5. bbum says:

    Documentation clarification is tracked in rdar://problem/7212642. If that is addressed, you’ll be officially wrong, too. ;)

    I didn’t suggest +load should be used. I stated the opposite, actually.

    The issue is one of fragility. The example I gave is, as stated, contrived, but it is exactly the kind of pattern I have encountered time and again when finding and fixing bugs across many applications. In all cases, the “heavy lifting” was initializing some random sub-system that ended up spawning threads and/or making network connections. Totally bad design, for sure, but that wasn’t how the code started — it evolved into doing that as features were added to the subsystem.

    The developer is far better off limiting +initialize to simple global state initialization and doing any sub-system initialization either at launch (if always needed) or on load of whatever needs said sub-system.

    Therein lies the problem. +initialize starts out simple, doesn’t do heavy lifting, but then requires more and more stuff to be integrated into it as soon as you stop just doing simple stuff.

  6. links for 2009-09-10 | /dev/random says:

    […] bbum’s weblog-o-mat » Blog Archive » +initialize Can Be Executed Multiple Times (+load not so mu… if a subclass does not implement +initialize but its superclass does, then that superclass’s +initialize will be invoked once per non-implementing subclass and once for itself. (tags: iphonedev development objective-c osx cocoa) […]

  7. links for 2009-09-10 « Mike’s Blog says:

    […] bbum’s weblog-o-mat » Blog Archive » +initialize Can Be Executed Multiple Times (+load not so mu… if a subclass does not implement +initialize but its superclass does, then that superclass’s +initialize will be invoked once per non-implementing subclass and once for itself. (tags: iphonedev development objective-c osx cocoa) […]

  8. James says:

    bbum,

    Thank you so much for updating your post to make my point for me. :)

    Your code example is brilliant and illustrates my original comment perfectly. The problem here is in the implementation of the thread synchronization and has nothing to do with +initialize. Specifically, the use of a single NSLock in this design is fatally flawed. There are multiple methods using the same NSLock that can call each other recursively. That’s just wrong. Either they should be using separate locks, or an NSRecursiveLock, as appropriate.

    Bad code can’t be use as an inditement against +initialize. You could add this code to any other set of method, class or instance, and have the same problem. The code example that should have been supplied should look something like this:

    (note: code pasted into browser from a working Xcode project – might not run as copied)

    @interface TestRunner : NSObject
    + (void)testThread:(id)ignored;
    @end
    
    @interface OneClass : NSObject
    - (void)hello;
    @end
    
    @interface TwoClass : NSObject
    @end
    
    NSRecursiveLock *globalResourceLock = nil;
    static OneClass *oneCircular;
    static TwoClass *twoCircular;
    
    int main (int argc, const char * argv[]) {
        NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSLog(@"Hi!");
    	
    	globalResourceLock = [NSRecursiveLock new];
    	for (int i=0; i<3; i++)
    		[NSThread detachNewThreadSelector:@selector(testThread:)
    								 toTarget:[TestRunner class]
    							   withObject:nil];
    	
    	[NSThread sleepForTimeInterval:4.0];
    	[globalResourceLock release];
    	NSLog(@"Bye!");
    	
        [pool drain];
        return 0;
    }
    
    @implementation TestRunner
    
    + (void)testThread:(id)ignored
    {
    	NSAutoreleasePool *pool = [NSAutoreleasePool new];
    	
    	NSLog(@"Thread starting...");
    	OneClass *test = [[[OneClass alloc] init] autorelease];
    	[test hello];
    	
    	[pool release];
    }
    
    @end
    
    @implementation OneClass
    
    +(void)initialize
    {
    	NSLog(@"%s started",__func__);
    	[globalResourceLock lock];
    	twoCircular = [TwoClass new];
    	[NSThread sleepForTimeInterval:2.0];
    	[globalResourceLock unlock];
    	NSLog(@"%s done",__func__);
    }
    
    - (void)hello
    {
    	NSLog(@"%@ says hello",self);
    }
    
    @end
    
    @implementation TwoClass
    
    +(void)initialize {
    	NSLog(@"%s started",__func__);
    	[globalResourceLock lock];
    	oneCircular = [OneClass new];
    	[globalResourceLock unlock];
    	NSLog(@"%s done",__func__);
    }
    @end
    

    Here’s an application the spans multiple threads, started currently, that all initialize the same class, which initializes another class, that recursively uses the first class. This should be the perfect storm for +initialize to lock up our application. But, of course, it runs perfectly:

    2009-09-12 09:19:20.798 InitializeTest[2180:10b] Hi!
    2009-09-12 09:19:20.803 InitializeTest[2180:1403] Thread starting...
    2009-09-12 09:19:20.803 InitializeTest[2180:1503] Thread starting...
    2009-09-12 09:19:20.803 InitializeTest[2180:1603] Thread starting...
    2009-09-12 09:19:20.804 InitializeTest[2180:1403] +[OneClass initialize] started
    2009-09-12 09:19:20.805 InitializeTest[2180:1403] +[TwoClass initialize] started
    2009-09-12 09:19:20.806 InitializeTest[2180:1403] +[TwoClass initialize] done
    2009-09-12 09:19:22.806 InitializeTest[2180:1403] +[OneClass initialize] done
    2009-09-12 09:19:22.808 InitializeTest[2180:1403]  says hello
    2009-09-12 09:19:22.809 InitializeTest[2180:1503]  says hello
    2009-09-12 09:19:22.809 InitializeTest[2180:1603]  says hello
    2009-09-12 09:19:24.803 InitializeTest[2180:10b] Bye!
    

    So I return to my original assertion: If +initialize is locking up your application, then your thread synchronization is flawed. Solving the problem by avoiding +initialize is simply avoiding the bug in your code. It doesn’t fix the real problem, and the problem has nothing to do with +initialize.

  9. bbum says:

    James — you are correct in that you can jump through hoops to design and implement your code such that it can work in the unpredictable, unordered, world of +initialize invocation. And my code, as stated, was a contrived example — it was a simple single threaded example of the multithreaded pattern that leads to deadlock.

    That your code demonstrates that it is possible neither makes it correct, nor a good idea.

    The bottom line is that +initializes are executed in arbitrary order and there are locks involved. If you cause complex sub-systems to be initialized from +initialize, your code will be fragile. It might work on your system, but not on a system with a larger/smaller number of cores. It might work on 10.5.8, but it might not on 10.6.0. Or it might break between 10.6.1 and 10.6.2. You are setting your code up to be dependent upon the +initialize of ever other class in your runtime being equally as well behaved and ordered as your code.

    As I have said before, I have debugged many many situations where deadlocks have occurred because the developer was surprised by a sudden change in order of calls to +initialize or had naively introduced concurrency without realizing the subtle dependencies incurred by +initializes own locking implementation. In all cases, fixing the code moved the heavy lifting out of +initialize and into a much more controlled phase of application bringup.

    The resulting code was less fragile, easier to maintain, and often eliminated various bits of locking infrastructure since the initialization work was triggered by passing through a very well known and controlled phase in application bringup.

    There are much better phases in application launch or subsystem initialization to do this kind of work.

    Do not do heavy lifting in +initialize. It is fragile, prone to concurrency bugs that are difficult to debug, and, in the near future, the documentation will emphatically indicate that you shouldn’t do so.

  10. sas says:

    I think your (very interesting argument) resembles a bit Scott Myers’ Effective C++ Item 47: Use multiple inheritance judiciously. Yes, you can make MI work if you’re careful but invariably someone down the line will do something that will cause trouble, because your initial assumptions don’t apply anymore. If I understand correctly, even though the locking can be designed to properly work around the ordering issue, at some point (maybe much later) someone ignorant of that design could accidentally introduce potential dead-locking. With the additional danger that it might not even cause problems straight away (the worst kind, really). So in that sense doing heavy lifting in +initialize is fragile I guess.

  11. bbum's weblog-o-mat » Blog Archive » objc_msgSend() Tour Part 4: Method Lookup & Some Odds and Ends says:

    […] that during +initialize, methods won’t always be cached. Yet another reason to not do any real work during +initialize!In part 3, the cache lookup loop contained a NULL check and, if NULL was encountered in […]

  12. Brent says:

    Note that in non-gc, +load runs without an autorelease pool, and so you have to make your own, which is ugly. But all one-time initialization code ends up ugly to some degree, I guess. You’re always checking whether you’ve already done it, and you have to make sure you handle synchronization.

    Sometimes it’s best to just take shared data/code and put it into a different class, probably a custom singleton.

    I like the pattern of having a class method to retrieve this stuff; let it do what it has to do lazily and handle synchronization. Alternately, make sure you deliberately get your initialization out of the way before you go spawning threads. Granted if you’re writing a library, you won’t have control over that. I personally like to avoid any kind of cyclical dependency between classes if at all possible, and stick with hierarchies; they’re easier to understand. I don’t so much like Bill’s turning a rule-of-thumb into a flat-out commandment, but whether or not you can (or should) do heavy lifting in +initialize, there’s probably no overwhelmingly convincing reason to do so.

  13. Quora says:

    What are some useful categories for iOS development?…

    Answering the question with a warning… Be really careful with categories. While absolutely awesome (and one of the Objective-C features I love the most) categories can trip you up and cause your code to be cruftier and harder to maintain. * Up until …

  14. Objective C类方法load和initialize的区别 | 编程·早晨 says:

    […] +initialize Can Be Executed Multiple Times (+load not so much) […]

  15. Abhi Beckert says:

    It is also worth noting you don’t necessarily have to define the subclass yourself. Sometimes cocoa will dynamically create a subclass at runtime.

    For example, I had a subclass NSKVONotifying_MyClass trigger a second call to +initialize, on a class I was confidant did not (and never would) have any subclasses.

    (Ed: excellent point! Thank you!)

Leave a Reply

Line and paragraph breaks automatic.
XHTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>