This post’ll be all over the map — there has been a lot of traffic on this subject. This stuff is particularly interesting to me. I have been banging out object oriented code for more than 20 years in a variety of languages — Obj-C since 1989. Thinking about this stuff is my day job, too. That rocks. Makes me really happy.
Daniel Jalkut recently wrote a post entitled “C is the New Assembly“. I’m not entirely sure what rock Daniel has been under for the last 20 years, but I’m pretty sure that C has always been considered to be little more than a portable macro assembly language by many folk for a long long time. I certainly remember such analogies being made when I first learned C.
Well, when I first learned C in a non-perverse environment. I learned C writing hypercard plug-ins. Yes, I learned C writing plug-ins for an application that only offered a Pascal API in an environment where pointers were often really pointers to pointers (handles) where the pointer would occasionally move out from under you. Fortunately, I was too ignorant to know that I was learning C the hard way.
But I digress. C has always been in the role of the “fallback for performance reasons” solution as soon as developers started writing significant bodies of code in other languages. Perl based CGI always had random modules written in C. Java has always had a painful native interface. C++ developers have long written chunks of code in C (though that has grown less common over time). Even the scripting languages all have supported dynamic loading from the beginning that has always been used for both accessing native functionality and for implementing “performance critical” stuff in native C.
Update: Daniel commented on this post:
I compare C to assembly completely apart from its rather coincidental technical approximations to assembly. I meant it on a much higher level. On a level where I hope in 10 years to be saying “Ruby Is The New Assembly.” It’s a coincidence that C is a relatively good alternative to assembler for producing highly optimized code. It’s still the case that C programmers “drop into assembler” when necessary to squeeze extra performance out of a particular platform.
Python is “close enough to C” that I suspect somebody in 5 or 10 years could say “Python Is The New C” and have a bunch of critics chime in “it’s been the new C for 15 years!” Well, not in my universe it hasn’t.
Actually, I think C developers dropped into assembly to squeeze out extra performance (or do stuff that wasn’t possible in C) more often in the past than they do now. The reasons are two fold:
First, fewer developers can effectively develop in assembly anymore. Not because programmers are dumber these days, but because writing assembly on modern CPUs is One Hell Of A Lot Harder than it was back in the 6502 or 68000 days. As well, modern codebases tend to be more portable even when only targeted to one manufaturer’s CPU — programming for a G3 vs. a G4 vs. a G5 could require very different low level code, if you were trying to eek out every last bit of performance.
Secondly, optimizing compilers have improved by leaps and bounds over the years. In particular, the optimizing part of the compilers are truly amazing chunks of linear algebra that do all kinds of subexpression optimization, execution reordering, and all kinds of relatively non-obvious tricks to make the code go really fast on modern processors.
Rare is the programmer that can effectively write assembly code on a modern CPU. Rarer still is the programmer that can write assembly code that performs better than that generated by a good compiler.
I agree with Daniel. I sincerely hope that we get to the point where “drop into primitive python” is an answer to various problems we are having in our highly abstracted programming environments of the future.
To get there, I think we will first have to get past treating software development like a specialized word processing exercise.
Daniel said (which I ran into before I saw Jesper’s post, so I’ll respond to it here prior to Jesper’s post because, well, that was how my brain wen down this particular path):
He suggests that a typical developer will write everything in Ruby or Python, and then do performance testing. Anything that needs a speed-up can be redone in Objective-C. You know, that “slow” dynamic variant of C
This analysis is foreboding, because it’s exactly what programmers have always done when they switched to a higher level language.
Yup. It is exactly what programmers have always done. Lots of Objective-C programs have C or C++ engines under the hood for the “performance sensitive” parts. I put that in quotes because, more often than not, the optimization wasn’t necessary — it just added complexity for little or no actual performance benefit. Not always the case, but more common than anyone would care to admit.
And it is no different with Python or Ruby based GUI apps. Native code is commonly used for the performance sensitive parts. Of course, with PyObjC or RubyCocoa, much of the optimization-in-native-language has already been done because, after all, the python and ruby in said programs are typically just gluing together already existing native language frameworks.
If you think about it, a Cocoa app that is using — say — Core Data and Quartz Composer is consuming very little CPU in actual developer authored code. Most of the CPU time is spent in QC or CD either pushing data around or rendering graphics in 3D; 100s of thousands of lines of code being driven by relatively few lines of custom code found in the project.
So at the end of the day, writing Ruby Cocoa apps is going to be a whole lot better and easier than writing Java Cocoa apps
The problem with the Java Bridge is that it tries to make the Cocoa API look like a Java API. The end result is an API that doesn’t look like Cocoa and doesn’t feel natural to a Java programmer.
The RubyCocoa and PyObjC bridges don’t try to “mogrify” Cocoa into Ruby-esque or Pythonic APIs beyond a slightly syntactic adjustment to take the Objective-C interleaved method name and arguments style dispatch and turn it into the much more common call-it-like-a-function style dispatch. (No, for the last time, Objective-C does not have “named parameters”. Never did.)
As a result, you pretty much always know when you are message across the bridge except for really simple things where it generally doesn’t matter that much.
Jesper goes on to conclude:
Writing Java-Cocoa with some Objective-C is like trying to write some parts of your book in BritishInternational English and some parts in US English. Same everything, you just wrote ‘old chap’ in one chapter and ‘homeboy’ in the next. Writing Ruby-Cocoa with some Objective-C is like writing half your book in Swedish and half your book in English. The two are noticeably different, and Swedish kicks English’s ass in a variety of places, so you can actually play on their respective strengths.
This isn’t true. Or, at least, it is backwards.
Objective-C, Python and Ruby all use very similar class models. And all of them are quite a bit different than Java. Whereas Java is naturally statically typed and bound, the other three languages are naturally dynamic. Java’s introspection and dynamic dispatch is a complete pain in the ass. Introspection and dynamic dispatch come very naturally to the other three languages and are quite often leveraged across large bodies of code (Cocoa, Twisted, and Ruby on Rails all come immediately to mind).
As for bridging, connecting Python and Ruby to Objective-C is a much more natural fit than Java. Python and Ruby both allow for easy on the fly class definitions and, though the API is a bit primitive, so does Objective-C.
Because of the way Java works, you can’t really dynamically create class definitions on the fly without either generating byte code on the fly or calling out to the Java compiler. Even once you do, the impedance mismatch between the Java class library(ies) and Objective-C are downright painful. Hell, String is declared final — cannot be subclassed — and, as a result, bridging from NSString to java.lang.String pretty much requires making copies across the bridge. Even if copies weren’t really expensive, you then have to keep track of the various instances on either said for purposes of preserving identity. I could go on. And on. To summarize: Ouch.
From the comments on one of the posts, Mark Grimes says:
The script bridges also aren’t uniform. E.g. RubyCocoa and RubyObjC have different performance metrics when crossing the bridge. The former requires implementation constraints that dictate not crossing the bridge too often. But alas, I agree with Scott, the popularity of these languages along with the abandonment of the Java bridge will get people new to OSX on-board until they discover how much more versatile ObjC is.
Then what? You package the app and the interpreter as a part of an application bundle? Ewww.
First, RubyCocoa has been fixed. It is much more efficient now, using libffi like PyObjC to achieve pretty damned good performance across the bridge.
Objective-C more versatile? I’m not really sure that is a fair assessment. Each language has strengths and weaknesses. More importantly, each has access to sets of technologies that are way beyond anything available in other languages.
For example, whenever I write a Cocoa application that has to do heavy network communications, I immediately turn to Python via PyObjC for the bulk of the implementation. Why? Because I can use Twisted, which is — hands down — the best damned network programming toolkit around (steep learning curve, amazing framework — think of it as the AppKit for Networking). Nothing available in Objective-C can touch it (without writing a ton of code anyway).
Finally, packaging the interpreter and your custom modules into the app wrapper really isn’t that big of deal. At least, it hasn’t been a problem for the handful of commercial applications shipping today that are implemented using Python, Cocoa, and PyObjC. As a matter of fact, you would never know that they were written in Python unless you went poking about the app wrapper. Even the launch times aren’t noticeably slower, if various optimizations are employed (especially with modern machines). Size really isn’t an issue given that most polished commercial apps will include megabyte upon megabyte of artwork, image laden documentation, sample material, and other multimedia capabilities.
Something to keep in mind: Bridging to Cocoa from scripting languages is nothing new. This is a very mature field of technology. PyObjC has been around since 1994, being used scientific, financial and commercial applications throughout its 13 year history. There were scripting bridges before PyObjC (I wrote a fairly full featured Tcl bridge at one point, for example) and, obviously, several others that were created after.
And it hasn’t just been for casual hacking. There are people who rely upon said bridging technologies every day. If the bridges don’t work or can’t be relied upon in the customer’s hands, it would be a major and often costly problem!
Jens Alfke commented:
I haven’t tried to develop a GUI app in a scripting language (Ruby or Python) yet, though I’ve used both for web apps, and have done tons of GUI development in Obj-C, Java and C++.
One problem I suspect will bite people using scripting-language bridges is the lack of type checking. Yes, I’ve read all the propaganda about “duck typing” and how the lack of type checking makes you more Agile. For small quickie bits of code, I tend to agree. However, in a full app, type errors will bite you in the ass. A lot.
In practice, type errors don’t really cause any more problems with GUI apps written in scripting languages than they do with carefully compile-time type checked Objective-C Cocoa apps. Unless, of course, you try to treat the untyped language like a strongly typed language or vice-versa. It is really a question of architecture — don’t try to write Python code that tries to manage types like a C++ program!
There will still be type errors, but they are mostly caused by various dynamic behaviors such as incorrectly ripping apart an NSNotification’s userInfo or improperly traversing the DOM tree of some XML gunk or the like.
I have worked on and have worked with developers working on some pretty good sized Cocoa apps that have significant chunks implemented in Python via PyObjC. Type errors really weren’t any more or less of a problem than with compiled Objective-C.
And, yes, I completely agree that automated testing of GUI apps is nearly impossible. Scripted based GUI development does offer one significant advantage. In particular, Python and Ruby make it quite natural to “dynamically load” additional functionality, both have easy to use unit testing facilities, and both languages make it easy to inject functionality into existing code.
As a result, you can do some fun in-context unit testing in conjunction with the user based testing. Or you can set up testing such that the user can play around with the app and then hit a button to have a bit of testing code injected that can verify the internal state of the app.
All stuff that can be done in non-scripted GUI apps. No question about it. It just somehow seems easier and more natural in Python and Ruby. Maybe it is just me.
Bret says something really funny:
Uncompiled languages will never take off for commercial app development. Here’s why: you are shipping your source code. OSS, server stuff, yeah – but not desktop apps the way they are defined now.
Pixar, Sandia Labs, several trading houses, and a number of other software development shops are all using PyObjC or RubyCocoa every day to help build their products, some of which include shipping said scripting bridges as a part of their product suite.
You youngsters will eventually “discover” Common Lisp.
Ya know… there is a Lisp to Objective-C bridge….