Archive for the 'Code' Category

PyObjC 1.3.6 is now available.

Wednesday, May 18th, 2005
DogSnake

Update: PyObjC 1.3.5 contained a bug that prevented plug-ins from working correctly. It has been fixed in 1.3.6.

PyObjC 1.3.6 is now available for download. This release adds support for many new features in Tiger, maintains backwards compatibility with Panther/Jaguar, and fixes many important bugs.

See the NEWS file for full details.

Some highlights:

  • The Xcode templates now use py2app. This means that development builds usually take less than a few seconds for a build from clean and considerably less than a second for incremental builds. Deployment builds will automatically create drag-n-droppable products for easy distribution.
  • in and inout parameters are fully supported. This includes the ability to pass NULL to the underlying API.
  • Core Data is fully supported. This includes py2app automatically compiling Core Data models from their development form into the runtime form.
  • A decorator is provided that allows easier specification of method signatures when using Python 2.4.
        @objc.signature(‘i@:if’)
        def methodWithX_andY_(self, x, y):
            return 0
  • NSData, NSDecimalNumber, and NSDate are now converted to appropriate native Python types automatically. serializePropertyList() and deserializePropertyList() can be used to convert an Obj-C property list to an NSData and back.
  • The proxy for pure Python objects acts more like a proper NSObject than before. This allows objects returned from pure Python libraries to be used from Objective-C in a more transparent fashion, including support for KVC.

Overall, this release really adds a tremendous amount of polish to PyObjC. Everything is very well tested and the behavior is just more consistent. Py2app has evolved significantly, as well, gaining Tiger specific features and many other refinements.

Great stuff. A huge thank you to Ronald and Bob. PyObjC is a production quality development tool that is truly a joy to work on and use.

So why doesn’t Java matter on the desktop?

Monday, April 25th, 2005

Yeah, someone busted me on actually failing to answer that question. So, a highly opinionated set of my observations.


In the interim years between which the Java promise was made and broken (again I’m only talking about the shrink wrap Application market here), there have been a number of other tools and technologies that have come along to make the whole WORA point not terribly interesting.

Beyond the development tools on the major platforms maturing significantly, there is also better support for building cross platform non-GUI code. Scripting languages are more easily integrated now than ever before, creating a high level portable solution for implementation of a good chunk of applications. Similarly, the support for C++ (and, really, Objective-C) in cross-platform environments has matured significantly in that write-once-run-anywhere-with-the-right-compiler-flags is a commonly used paradigm these days.

As well, the approach with which software is developed has matured. Developers recognize the value of separating user interface code from their data containment code. Model-view-controller is an obvious and commonly used design pattern that does exactly that. There are other design patterns that achieve the same goal. It is also interesting to note that the growing popularity of unit testing naturally leads to a better separation of user vs. non-user functionality simply because it is so difficult to unit test user interfaces (that is what functional and acceptance testing is all about).

And the tools available for separating data from user interface have also matured significantly. Beyond design patterns, there are now a number of tools that vastly simplify client/server style communications (even when a network connection isn’t really involved– SQLite, for example). Use of client/server also naturally leads to a stronger separation between the platform specific user interface code and the platform agnostic data management and core functionality.

So, while the Java market was largely focused upon providing solutions– some awesomely powerful– for developing internet related or internal/vertical applications, the rest of the market was not sitting around.

Likewise, the platform specific development models have advanced significantly. On Mac OS X, Windows, and Linux/BSD/X, there are many relatively new tools and APIs that make pumping data into a user interface both a heck of a lot simpler (fewer lines of code) while also enabling the ability to create custom visualization tools that are strongly tied to the native APIs. Certainly, Java is providing support for a similar mechanisms via some of the newer APIs, but the development experience still lacks the tightly bound fluidity of using tools designed specifically for the platform.

There have also been a number of APIs and technologies built that also promise write-once-run-anywhere style solutions for cross platform deployment. While all suffer from similar look and feel problems as Java based solutions, there are some very high quality solutions in this space that often have significant advantages over Java — smaller installation footprint, less overhead, simpler development, or some combination therein.


So, as a result of both the significant advancements in platform specific technologies and in the cross-platform tools market, Java is simply yet another solution to an already solved problem.

All of the above combined means that Java just won’t matter on the desktop unless it can offer a compellingly superior solution to some set of problems. At the moment, Java does not.

The Java specific IDEs are really powerful, but none offer truly compelling user interface development tools.

While writing cross-platform non-UI specific code is both easy and quite attractive with Java, cross-platform GUI code really doesn’t exist without sacrificing the quality of UI that can only be had with native tools. That, combined with the increasing quality and simplicity of tools such as wx*, TK, or a number of commercial products (Real Basic, Runtime Revolution, etc..) for cross platform development further dilutes Java’s potential.

Given the discussed roadmap of Java these days, it doesn’t look like Sun — or anyone else — is really putting up much of a fight for the desktop.

Followup: Why Java on the Desktop Just Doesn’t Matter

Monday, April 25th, 2005

That last post certainly deserves a followup. It generated quite a bit of commentary, some of which was interesting and most of which indicated that the author didn’t actually read what I wrote (or I did a poor job of conveying what I intended).

In any case, thanks to all that responded. It was a remarkably flame/fud/troll free set of responses for a topic that is generally quite volatile in nature.

Some random points of response:

What Carmack Said I know that Carmack’s comments were directed at development of software for cell phones and other embedded devices. The point of my post was to take this particular observation and apply it to what I perceive as a significant failure in regards to the original promise of Java. Java spent a good chunk of the latter part of the 90s being marketed as the end-all, be-all, write-once-run-anywhere, solution for application development. That promise never materialized.

Java, the Language I like Java, the language. I have written tons of Java. It was mostly pleasant, mostly interesting, and often quite entertaining. About the only bits of Java I really found miserable had to do not with the language, but with dealing with other people’s code that was exceptionally poorly written. Java certainly does not own a monopoly on crappy code. However, as a result of so many really bad ideas getting funding during the dot-bomb years, many bits of code that should never have seen the light of day were shipped as “final” products. Until working for Apple, much of my professional time was spent cleaning up other people’s messes. While lucrative, it also meant that I spent a large amount of time de-tangling really crappy code from a client’s environment, often well after that client had spent a boatload of $$$ on said solution. Often, it was a better bet to put up with the crappy code to make the client feel like they didn’t get completely ripped off than it was to do the right thing and throw the garbage out.

One of the promises was that Java development would be efficient. We have seen tremendously powerful features come forth in the various IDEs, but one area that seems entirely lacking is the UI development tools. This is probably one of the reasons why Java as a desktop application development environment (where, again, “desktop application” is “stuff you’d see in a shrink-wrap box at CompUSA) just never met the promise. The tools for building applications sucked when the promise was made, and continued to suck until long after the promise was broken. I have started to see some promising tools in the last couple of years, but none that seem really aimed at building generic applications — most seem to be either tied to some enterprise related back end framework or to solving a particular niche problem such as reporting.

And, yes, Java is not slow. I know that. I wrote many a hunk o’ Java code that had to be fast as hell and it was always possible to eek out the performance I needed. But Java desktop apps, on the other hand, generally are very slow, or unresponsive, or bloated pigs, unless the developer expends an unbelievably huge amount of effort to make them otherwise.

Real World Java Desktop Apps Java promised write-once-run-anywhere style development where the end result would look-n-feel like a native, or near native, app on the target system. That promise was quickly amended to account for that fact that different desktop environments behave differently and some of those differences would require compensation within the codebase to ensure a native look and feel. This is certainly not a flaw of Java, but merely the reality of building desktop apps. However, even that is not really true. Achieving a native look-n-feel while also creating a responsive application — one that doesn’t “feel” any different than any other app on the platform — just doesn’t seem to have happened very often.

What many people missed was that I was specifically talking about commercial applications targeted to the wide market. Not to niches, not development tools, but addressing the original promise that Java would be a major player in the applications market. Sure, many many people are using Java quite successfully to build in-house, niche, one-off apps. I was in that group for a long time and assuredly will write new Java one-offs for my own or other internal uses in the future. Visual Basic has achieved an even greater success in such a role. But that wasn’t the promise! The promise was that Java would be the solution for creating desktop applications of the kind you would see in shrink wrap boxes at the likes of CompUSA or Amazon.com.

So, of course, a number of people responded to the original comments by pointing out various “example” applications that are written in Java, have achieved some level of widespread distribution, and are targeted to the desktop environment.

Azureus, LimeWire Actually, I used Azureus pretty regularly when grabbing etree content. Painful user interface, not very responsive, and made no attempts to look like a native app. But, yes, very very powerful and did a wonderful job of what it did. Certainly not something I would call a “commercial quality” application. But, then again, my standards– having come from a NeXT and Mac background– may be exceptionally high. LimeWire, last I looked, was a similar kitchen sink’s worth of powerful features shoved into a single app with little thought for usability. Both are bloated, too — consume huge amounts of memory.

Java IDEs Yup — there are a number of IDEs written in Java that are awesomely powerful. Java lends itself really well, much better than Objective-C, to building things like complex and very powerful refactoring tools. As a result, the Java IDEs can do some really amazing stuff with code.

But, boy, talk about bloating pigs! IDEs, in general, are typically 10s, if not 100s, of megabytes for the app along + often 100s of MB more for the supporting materials. And there is no easy way to approach the IDE. You are pretty much left with “run the app and hunt until you figure out how to create a new project… then a new file… then to build… then to debug…etc…”. Worse, many of the IDEs seem to have invented their own UI paradigms that guarantee that the look-n-feel is going to be far away from anything native and often quite a bit aside from the “generic Java” l-n-f.

So, while I consider the Java IDEs to be a testament to the power of Java and to the maturity of certain segments of the Java market, they are certainly not examples of “wora, looks great, native look-n-feel, shrinkwrap package on a shelf” type apps!

Quantrix Someone mentioned Quantrix, which has been ported to Java. If it works half as good as the original Improv, that is one hell of an achievement and they should be applauded. However, given pricing it would seem to be relegated to a vertical market or niche at this time.

Enough of this…

Why Java on the Desktop Just Doesn’t Matter.

Tuesday, April 12th, 2005

John Carmack has recently pursued software development targeted to cell phones. In his typical fashion, he posts about the realities of software development on cell phones.

What caught my eye were his comments on the use of Java for the development of interactive user interfaces. In particular, John says

“The biggest problem is that Java is really slow. On a pure cpu / memory / display / communications level, most modern cell phones should be considerably better gaming platforms than a Game Boy Advanced. …. there is just no excuse for having 10+ millisecond granularity in timing. Given that the java paradigm is sort of thread-happy anyway, having a real scheduler that Does The Right Thing with priorities and hardware interfacing would be an obvious thing. Pressing a key should generate a hardware interrupt, which should immediately activate the key listening thread, which should be able to immediately kill an in-process rendering and restart another one if desired. The attitude seems to be 15 msec here, 20 there, stick it on a queue, finish up a timeslice, who cares, right?

There was a period of time when the promise was that Java would take over the desktop. Every company was focused on either “standardizing” on Java as their language of choice for desktop development/deployment or was fighting like hell to try and dilute the “write once, run anywhere” promise of Java to maintain their proprietary stranglehold on a particular piece of the market. All the while, Sun milked the hell out of it, spending massive marketing dollars perpetuating the promise.

But that promise has certainly never been realized. And, now, it looks like it never will. Can you name a single commercial desktop application that is implemented largely in Java? I can’t. Sure, there are a number of various random products that are dominating particular niches. But when I look at the boxes of software in CompUSA or an Apple Store, I don’t think I have seen a single piece of popular or “major” software written in Java.

What John said above is largely why. Java is a great general purpose language. I have written tens of thousands of lines of Java code for various random web and web related applications and enjoyed doing so quite a bit. If you remember, I was one of the handful of developers in the WebObjects community that fully embraced the move to Java.

Java has failed on the desktop because the architecture does not yield a decent and responsive user experience without a tremendous amount of effort fighting against the “natural” patterns of the various toolkits. The promise of Write Once, Run Anywhere mostly– well, pretty often– works for non-UI applications. For UI applications, the differences between platform and UI philosophy mean that WORA also yields a user experience that is alien to the platform.

Likewise, the very architecture of Java naturally incurs latency throughout any Java realized UI. As John said “15msec here, 20 there, stick it in queue … who cares?” For UI development, that is a deal killer.

Unit Testing article at ADC

Tuesday, March 29th, 2005

The Apple Developer Connection has published an article covering Unit Testing of Objective-C with OCUnit.

I have been using OCUnit for quite a while to tremendous benefit. Having OCUnit integrated into the development process in such an intimate fashion has resulted in much higher quality products with a lot less churn– integration churn, regression churn, cluelessness churn– and, overall, significantly more pleasant development cycles.

While I choose to use OCUnit for a variety of reasons, not the least of which is that I have been using it for many years, the recently (by comparison) released UnitKit is also quite nice. And, as Duncan said, “If you don’t test with UnitKit [or OCUnit, TestKit, JUnit, PyUnit, etc..] please at least test. Unit testing is probably the single most effective thing that you can do to improve the quality of your code.”

Speaking of PyUnit, you can also unit test Objective-C from Python via PyObjC. Jiva DeVoe wrote an excellent article on doing just that. Python based unit testing have the advantage of an extremely quick turnaround time, but suffer from the inability to debug in the same language that the code being tested is written in.

Good stuff.

httpflow; tcpflow parser to help debug http

Wednesday, March 16th, 2005

I ran across this article mentioning something called httpflow and was momentarily confused as I co-authored something called httpflow at one point.

It is still useful, so a quick rehash (to update the Google cache):

A really long time ago, Ben Holt and I wrote a little python script that parses the output of tcpflow and reconstructs HTTP conversations from the raw data. Furthermore, it can easily be configured to detect and display or exclude specific headers as well as optionally squelching the body of the HTTP request/repsonses.

It was written because we were faced with a really nasty bug where certain cookies were being arbitrarily dropped. It turned out to be a bug in Apache’s mod_track.

The script is still available as a part of the sourceforge tcpflow project. I have also added it to my repository of hacques.

The original article:

Quite some time ago, Ben Holt and I wrote a useful little Python script that can parse the output of tcpflow (available in Fink as well as via just about every Linux package manager around) captured HTTP sessions and reconstitute them into actual requests and responses. We were motivated by the need to track down a handful of bugs related to Cookie handling in some WebObjects applications we were working on (turned out to be a bug in an Apache module) and the best way to do that is at the wire.

Well, of course, if you have ever captured every byte of a raw HTTP conversation, it is only slightly less painful than wading through the reams of raw data produced by simply capturing every bloody packet that goes across the pipe. Since most browsers are threaded, there was also the challenge of putting everything back into sequence.

So, we gradually added some options to the script that would allow one to filter for particular headers, turn on/off dumping of the bodies, and output to indicate which request caused a particular response to be generated.

In the end, this proved to be far easier to do than reconfiguring the machine to use some random logging/analysis proxy (which sucked at the time anyway) or otherwise disrupting our development environment.

It is a total hack and completely dependent on the format of the output of tcpflow. It has proven to be extremely useful in many situations over the years.

Recently, I wanted to capture the sequence of URLs passed from a web services client to a server.

So, this version of httpflow.py adds the ‘–terse’ option which will cause httpflow to emit only the GET/POST/HEAD/*method* line from the request/response. Very useful for seeing the sequence of URLs fired against a [known] server.

Example captures of searching for “fried catfish” via Google (including loading www.google.com) follow.

Terse:

% sudo tcpflow -c -p -i en0 | ./httpflow.py --terse
HTTPFlow Running (--help for usage/help)...
tcpflow[3160]: listening on en0
GET / HTTP/1.1
GET /images/logo.gif HTTP/1.1
GET /search?hl=en&ie=ISO-8859-1&q=fried+catfish&btnG=Google+Search HTTP/1.1
^Ctcpflow[3160]: terminating

Default:

% sudo tcpflow -c -p -i en0 | ./httpflow.py
HTTPFlow Running (--help for usage/help)...
tcpflow[3162]: listening on en0
--- begin header ---
Source: 192.168.001.070 : 55937 (-unknown-)
Destination: 216.239.039.099 : 80 (-unknown-)

GET / HTTP/1.1
Host: www.google.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/73 (KHTML, like Gecko) Safari/73
If-Modified-Since: Wed, 30 Apr 2003 04:44:52 GMT
Accept: */*
Accept-Language: en-us, ja;q=0.33, en;q=0.67
Cookie: PREF=***secret stuff deleted***
---- end header ----

--- begin header ---
From request: GET / HTTP/1.1
Source: 216.239.039.099 : 80 (-unknown-)
Destination: 192.168.001.070 : 55937 (-unknown-)

HTTP/1.1 304 Not Modified
Date: Wed, 30 Apr 2003 04:45:12 GMT
Content-Type: text/html
Server: GWS/2.0
Content-Length: 0
---- end header ----

--- begin header ---
Source: 192.168.001.070 : 55937 (-unknown-)
Destination: 216.239.039.099 : 80 (-unknown-)

GET /search?hl=en&ie=ISO-8859-1&q=fried+catfish&btnG=Google+Search HTTP/1.1
Host: www.google.com
Connection: keep-alive
Referer: http://www.google.com/
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/73 (KHTML, like Gecko) Safari/73
If-Modified-Since: Wed, 30 Apr 2003 04:44:59 GMT
Accept: */*
Accept-Language: en-us, ja;q=0.33, en;q=0.67
Cookie: PREF=
---- end header ----

--- begin header ---
From request: GET /search?hl=en&ie=ISO-8859-1&q=fried+catfish&btnG=Google+Search HTTP/1.1
Source: 216.239.039.099 : 80 (-unknown-)
Destination: 192.168.001.070 : 55937 (-unknown-)

HTTP/1.1 200 OK
Date: Wed, 30 Apr 2003 04:45:20 GMT
Cache-control: private
Content-Type:
Transfer-Encoding: chunked
Server: GWS/2.0
---- end header ----

Not pretty, but works well enough….

Getting started with Python on Mac OS X

Tuesday, February 22nd, 2005

A friend asked me what he should install onto his Mac OS X system to most effectively learn Python. This particular person is a very experienced Objective-C and Java developer, with loads of Mac OS X specific adventures.

This post is targeted to that kind of developer.

Mac OS X ships with a perfectly usable build of Python included in the system (as long as you install the BSD package, which is enabled by default).

First, install the most recent production release of the Subversion client. That link leads to some very convenient to install packages. Alternatively, installation via Fink or DarwinPorts will also work quite nicely. Or you can build it from the source.

Next, grab a pre-compiled copy of the readline.so module. Decompress it, then copy it into the appropriate place within /Library/Python/2.3/. Interactive Python without readline is miserable.

Then grab the top-of-tree of the PyObjC repository via Subversion. The top-of-tree is almost always very stable as a result of the relatively large and high quality unit testing suite and the focus-on-quality of the developers. Once installed, the benefits are two fold. First, it will install PyObjC, including all examples, the Xcode templates, and the runtime itself. Secondly, it will install py2app which will allow one to easily package Python modules and applications, including creating Installer packages out of any standard Python module.

svn co http://svn.red-bean.com/pyobjc/trunk/pyobjc/

And build/install it:

python setup.py bdist_mpkg --open

The above will build a .mpkg that contains PyObjC, py2app, and related resources, then open it the Installer application.

From there, it is a matter of choosing your favorite editor. Xcode, SubEthaEdit and Emacs can all edit Python quite effectively. The key is to turn off tabs. Never, ever, insert tabs into Python source — always use spaces.

There are other editors available. I pretty much stick to a mix of Emacs with a custom setup (you’ll want to grab the latest python-mode, at the least) and Xcode, with the occasional bout of SubEthaEdit, depending on mood.

For learning Python, I would start with Dive Into Python and then– because this is aimed to Mac OS X developers– move on to ripping into the PyObjC examples. Also, ReSTedit is a fairly decent sized Cocoa-Python application that is under active development, tends to track the latest changes in PyObjC, and has a couple of relatively complex third-party Objective-C classes integrated into the project.

That should provide a pretty decent foundation for both generic and Mac OS X specific Python development. Certainly, there are any number of other tools that one might install, varying in size and complexity.

Recommendations, corrections, and suggestions welcome.

Big improvements to PyObjC

Tuesday, January 18th, 2005

Bob made a bunch of improvements to PyObjC. Go read the post for details. Some very cool stuff.

PyObjC 1.2 Released

Wednesday, December 29th, 2004

PyObjC PyObjC v1.2 is now available. You can grab it from the sourceforge download page or from my idisk.

The major changes are listed below. There is some seriously cool stuff in here; new packaging, support for pure-python loadable NSBundles, categories, better bridging of complex types, and revamped examples.

  • PyObjCTools.AppHelper.stopEventLoop will attempt to stop the current
    NSRunLoop (if started by runConsoleEventLoop) or terminate the
    current NSApplication (which may or may not have been started by
    runEventLoop).
  • This version no longer support Python 2.2. Python 2.3 or later is
    required.
  • It is now possible to use reload on modules containing Objective-C
    classes.
  • objc.loadBundle now returns bundle we just loaded.
  • Added objc.loadBundleVariables and objc.loadBundleFunctions,
    two functions for reading global variables and functions from a bundle.
  • objc.runtime will now raise AttributeError instead of objc.nosuchclass_error
    when a class is not found.
  • objc.Category can be used to define categories on existing classes:
    class NSObject (objc.Category(NSObject)):
        def myMethod(self):
            pass
    

    This adds method myMethod to class NSObject.

  • py2app is now used for all Example scripts and is the recommended method
    for creating PyObjC applications.
  • Proxies of dict, list, and tuple now respect the invariant that you should
    get an identical instance if you ask for the same thing twice and the
    collection has not been mutated. This fixes some problems with binary
    plist serialization, and potentially some edge cases elsewhere.
  • There is now a __bundle_hack__ class attribute that will cause the PyObjC
    class builder to use a statically allocated class wrapper if one is
    available via certain environment variables. This functionality is used
    to enable +[NSBundle bundleForClass:] to work for exactly one class from
    a py2app-created plugin bundle.
  • We now have a working Interface Builder palette example due to
    __bundle__hack__.
  • bool(NSNull.null()) is now false.
  • setup.py supports several new commands:
    * build_libffi
    
      builds libffi (used by build_ext)
    
    * build_html
    
      builds html documentation from ReST source
    
    * bdist_dmg
    
      creates a disk image with the binary installer
    
    * bdist_mpkg
    
      creates a binary installer
    
    * test
    
      runs unit test suite (replaces Scripts/runPyObjCTests
      and Scripts/runalltests)
    
  • PyObjCStrBridgeWarning can now be generated when Python str objects
    cross the bridge by calling objc.setStrBridgeEnabled(False). It is
    HIGHLY recommended that your application never send str objects over
    the bridge, as it is likely to cause problems due to the required
    coercion to unicode.
  • The coercion bridge from Python to Objective-C instances can now be
    augmented from Python as it is exposed by OC_PythonObject. See
    objc._bridges. This is how the str -> unicode -> NSString
    bridge with optional warnings is implemented.
  • The coercion bridge between Python objects and Objective-C structures
    can now be augmented from Python as it is exposed by OC_PythonObject.
    See objc._bridges. This is how the Carbon.File.FSRef
    <-> '{FSRef=[80c]}' structure bridge is implemented.
  • Extension modules such as _objc, _AppKit, etc. are now inside
    packages as objc._objc, AppKit._AppKit, etc. They should never be
    used directly, so this should not break user code.

ReSTedit 0.50 Available

Wednesday, December 29th, 2004

screenshot

A build of ReSTedit 0.50 is available on my .mac file download page. It should “just work” on any Mac OS X 10.3 system as it includes both the pyobjc and docutils libraries within the app wrapper.

Beyond a handful of bug fixes, ReSTedit includes new features described in this post.

The README and ToDo list have also been updated.