Archive for the 'Code' Category

Prettifying Python’s unittest output (and making it more informative)

Sunday, October 29th, 2006

Yeah, I still write code. I just can’t publicly talk about it. Yet.

Recently, I have been writing unit tests that involve passing source code through GCC and then running the resulting compiled binary. Given that PyObjC already does this for the unit tests of ffi, I started my test harness by ripping off that one.

My tests actually look for very specific warnings and failures from GCC. So, I augmented the script slightly to allow me to embed “expect this warning/error” into the source to be compiled.

When such a test fails, it is terribly useful to actually see the command line that was invoked. Unfortunately, it doesn’t appear that Python’s unittest TestCase class has a hook for composing the failure details. But it does have a hook for easily setting the class of the exception that will be raised. From there, it is easy enough to pass a reference back to the TestCase into the various assert methods to allow a custom exception class to provide more info.

Don’t bother trying to parse that paragraph, just read the code:

class TestCaseAssertionError (AssertionError):
    def __str__(self):
        if len(self.args) is 2:
            msg, testCase = self.args
            return "\\n\\nCompiled using:\\n\\t%s\\n\\n------ Details ------\\n%s\\n------ End Details ------" % (testCase.commandline, str(msg))
        else:
            return str(self.args)

class DgTestCase (unittest.TestCase):
    failureException = TestCaseAssertionError
    def __init__(self, filename):
        unittest.TestCase.__init__(self)
        self.filename = filename
        self.commandline = None

    ... at some point, the command line is composed and self.commandline is set ....

    ... then, in a test case ....
    self.assertNotEqual(actual.strip().find(expected.strip()), -1,
    ("Expected to find:\\n\\t%s\\nNot found in:\\n\\t%s" % (expected.strip(), actual.strip()), self))

Easy enough. Now, when a compiler failure occurs, I can easily copy/paste the command line to see the full output or deduce the issue further. Though the above is completely specific to my needs it would be easy to generalize.

MarsEdit vs. PHP: dropping ‘class=’ from posts

Tuesday, January 10th, 2006

MarsEdit 1.1 is out. It adds a number of nice new features and bug fixes. I use MarsEdit to edit all my weblog posts. It is a great tool.

Coincidentally, WordPress 2.0 was also recently released. WordPress is a nice enough weblog publishing engine. For me, the most important features were decent comments, useful permalinks, and a lot of canned templates so I didn’t have to torture myself any more than necessary with debugging random software. “It just works” was my mantra in choosing WordPress.

Of course, it doesn’t always “just work”. In particular, since upgrading to 2.0, posts made through MarsEdit that included class= attributes on div tags — what I use to float images left and right (amongst other things) — were being blown away.

Yuck.

After wading into some PHP code (GROSS! What a horrible piece of coding torture that was), the problem proved to be relatively easy to “fix”.

The file wp-includes/kses.php implements a filter that strips all posts of nefarious bits. Unfortunately, it doesn’t let the class= attribute through on div tags.

The fix is easy, even if it most be made in about a 2k character long single line of code.

Just add class => array() to the div tag specification:

'div' => array ('class' => array(), 'align' => array ())

There is a MarsEdit forum discussion and a WordPress bug.

Advanced Mac OS X Programming

Thursday, November 3rd, 2005

Big Nerd Ranch recently released Advanced Mac OS X Programming.

This is an updated, second edition, printing of the book originally titled Core Mac OS X and Unix Programming. Aaron was kind enough to send me a copy of the original book of which I I wrote a review.

I also took the original edition to work. Last I saw it, it was looking rather dog eared from all the use. Now it seems to have disappeared.

Aaron was kind enough to send me a copy of the updated version.

This version is every bit as good as the first and it has been updated throughout for Tiger specific APIs and technologies. There are a couple of additional chapters and lots of refinements.

What makes this book so good?

Take fork(), for example. The fork() man page provides an extremely terse description of exactly what fork() does. As is typical, it gives absolutely no details or discussions as to the implications of fork()ing a Cocoa application or any of a number of other questions that any app level developer might need.

Core Mac OS X and Unix Programming has a discussion of fork() that spans several pages, includes details on the implications for Cocoa programs, inventories various caveats and related functionalities, and does all this while also providing several examples demonstrating the behavior provided in the text.

The first edition saved me many hours of research while often answering questions that I didn’t realize I needed to ask. I’m confident that the updated version will do the same. I certainly learned a thing or two about fork() in writing this blog entry!

The table of contents is available as a PDF. There is also a site for notes, errata, and source code.

If you are new to Cocoa programming, this book is not for you. There are a number of excellent books that cover basic to advanced Cocoa programming. As far as I can tell, Advanced Mac OS X Programming is unique within the Mac development book market.

Unit Testing and Code Coverage in Xcode

Saturday, October 29th, 2005

Chris Hanson and I both intended to write something like this someday. Now we don’t have to.

Go read this article about unit testing and code coverage. Specifically, Chris gives detailed instructions on how to integrate gcov based code coverage into Xcode while also leveraging Xcode’s support for unit testing.

Excellent article. Thanks for writing it, Chris. I hope you don’t mind if I file a couple of bugs against Xcode to remind us to integrate something like what you have done sometime in the future.

Tracing Python Execution

Thursday, October 27th, 2005

While working with Twisted and Nevow, I ran into a situation where something terribly nasty was happening well below any code I had written. Basically, I was faced with the classic high level development problem of “something I did long ago broke something deep within several layers of large scale systems I did not write”.

In these situations, a debugger does little good because there is little clue as to where things went awry. What I really needed was a way to trace execution.

It turns out that Python has excellent hooks for doing exactly that.

Add the following to any random hunk o’ code:

import sys
import linecache
import inspect

def traceit(frame, event, arg):
    if event == 'line':
        lineno = frame.f_lineno
        if '__file__' in frame.f_globals:
            filename = frame.f_globals['__file__']
            if (filename.endswith('.pyc') or
                filename.endswith('.pyo')):
                filename = filename[:-1]
            name = frame.f_globals['__name__']
            line = linecache.getline(filename, lineno)
        else:
            name = '[unknown]'
            try:
                src = inspect.getsourcelines(frame)
                line = src[lineno]
            except IOError:
                line = 'Unknown code named [%s].  VM instruction #%d' % \
                    (frame.f_code.co_name, frame.f_lasti)
        print '%s:%s: %s' % (name, lineno, line.rstrip())
    return traceit

Then, to turn it on, call sys.settrace(). You can easily toggle the trace functionality, thus reducing the tracing to a very limited portion of the overall execution. This prevents one from drowning in data.

sys.settrace(traceit)

Note that the function has a frame object as an argument. That frame object allows for extremely deep introspection of the execution state, including variable state and just about anything else you would like to find out about the underlying virtual machine.

The original code was cribbed from the dalke scientific site. That article has an awesome explanation of the details of tracing. I just added some error checking and fallback code in the case where the original file/source cannot be determined. This enables the tracing functionality to work seamlessly within the context of Twisted.

It has already saved me a ton of time and I’m about to throw this permanently into my local Python library.

Neat PyObjC Trick

Wednesday, October 19th, 2005

I demoed the latest build of PyObjC at the recent CocoaHeads meeting in Cupertino.

The demo concluded with a “show all [most, really] the tricks at once” sequence.

  • How py2app, a part of PyObjC, will automatically create an installer package from a standard Python package
  • Injecting a live Python interpreter into a running application
  • Introspecting the application
  • Interacting with Core Data, including dealing with ** style parameters
  • Modifying a managed object and having KVO automatically fire on assignment

First, grab the top-of-tree source from the pyobjc repository:

svn co http://svn.red-bean.com/pyobjc/trunk/pyobjc/

Alternatively, you can mount that URL in the finder (click cancel on all the bloody attempts to write .DS_Store files) and cp -r the source to somewhere from the Terminal.

Then:

cd pyobjc
python setup.py bdist_mpkg --open

That will build PyObjC and open an Installer package that contains the runtime, documentation, examples, and Xcode templates.

Next, grab the OutlineEdit examples from /Developer/Examples/CoreData/ and run it.

Now, from the PyObjC source directory:
Read the rest of this entry »

Twisted API Documentation

Friday, August 5th, 2005

I have been spending quite a bit of time writing Twisted code. Twisted is just plain cool.

Playing a particularly cruel and twisted joke, the developers did not see fit to post a link to the current API documentation anywhere that I could find on the TwistedMatrix site. Of course, I browse said documentation on a regular basis while google-coding.

Of course, I could not find the damned link again and I had forgotten to bookmark it. LaunchBar’s voracious appetite for Safari’s history bites me again.

So that I will never lose it again: This is a link to the Current API Documentation for Twisted. That’s right. I would click the link to the current Twisted documentation if I want to know about ProcessProtocol, reactor, twistd, spawnProcess, Deferred, or the rest of the internet-y goodness of Twisted.

Using FileMerge with subversion

Monday, August 1st, 2005

I have long wanted to use FileMerge as the target for svn diff operations on Mac OS X. But I was just too damned lazy to bother.

Fortunately, Mike Ash was not so lazy. Unfortunately, Mike’s simple/elegant solution titles the two versions of the file “tmp” and “tmp.2”.

This works fine if you have a short term memory span of more than about 30 seconds. I do not. My short term memory is more like that of a goldfish.

Mike’s script will also fail intermittently in that svn diff deletes the temporary files as soon as the command diffing them completes. That is often before FileMerge reads the files on my system.

So, I wrote a much longer and far less elegant script that puts the name of the file and version number into the title bar while also leaving copies of the renamed files in /tmp/svndiffshm-UID forever.

You can download the script from my personal subversion repository at red-bean.

To use, you can pass the –diff-cmd command line argument to svn diff. Personally, I want FileMerge to come up all the time.

For that, edit ~/.subversion/config. Uncomment the [helpers] line and [uncomment and] set the diff-cmd line to point to the script. It has to be an absolute path (I ought to file a bug against subversion about that).

Works for me. I’m sure someone will point out a far simpler way to do this.

Update #1: Ken Anderson pointed out that the script wasn’t working when using the generic svn diff command against a file with local modifications. Yup. Sure didn’t. Now it does. The link above will always lead to the latest version.

Update #2: Mark Rowe kindly submitted a patch to enable compatibility between svndiffshim.py and SVK. Patch applied. Changes comitted. Thank you. I really need to take some time to grok SVK. It appears to solve a number of problems that I run into daily, including the need for opaque collections.

Importing Live Recordings into iTunes.

Sunday, July 24th, 2005

I have long downloaded live music from ETree and other sources of live recordings. I pretty much stick to soundboard quality recordings, though I’m perfectly happy listening to a good quality audience recording of an excellent performance.

I use Xact to decompress the shorten/FLAC recordings to WAV which are subsequently compressed using the Apple Lossless Encoder within iTunes.

The most painful part of the whole process was applying the metadata to the tracks. The album/artist information could be bulk applied, but individual tracks required copy/pasting the track name and remembering to update the track N of M count (or else the tracks would be alphabetized and restoring the order was terribly annoying).

Nothing a bit of AppleScript can’t fix. I wrote a little script that automates the process (there are two scripts in the zip — Enumerate Tracks does the trick).

The workflow is something like:

  • Import WAV files into iTunes and select all imported tracks
  • Apply universal metadata — band name, venue, year, # of discs, genre, etc…
  • Select tracks on first disc
  • Copy all the track names from the info file or the web site listing the tracks
  • Run Enumerate Tracks
  • Each show is an album, for all intents and purposes. I have chosen a consistent naming scheme such that I can easily type one string to search for all live recordings, all live recordings in a particular year, etc… Basically, each album is titled Live; YYYY-MM-DD; Venue; City; State (or some variation of city/state for international shows).

    If all goes well, Enumerate Tracks will apply the “track N of M tracks” metadata and the track names. Careful of those line endings!

Days between dates & our perception of time

Friday, July 8th, 2005

NSCalendarDate and NSDate make it generally trivial to work with dates, times, time zones, and all that rot. At least, they make it easier than just about any other date/time classes I have run into.

Still, there are a handful of operations that are non-obvious. At least, non-obvious to me.

One, in particular, is answering the question of “how many days between this day and that day?”. The obvious answer is to grab the time interval between the two dates and divide by the number of seconds in a day. The problem is that a date object stores a date and time. If the two dates represent, say, morning on one date and afternoon on the other, the difference in days between will be off by one.

The math is correct, but our perception of “days between” has nothing to do with the time of day between the two days. Capturing perception in an API is hard.

For example, most people consider tomorrow one day away even if it is 8pm today.

My first pass at implementing a solution was to derive calendar dates locked to midnight for the two dates and do the math. And it doesn’t work with generic NSDate instances.

Instead, a bit of basic math solves the problem nicely. The following is for calculating the # of days between a “due date” and “today”.

    int dueDateTimeInterval = (int) [[self valueForKey:@"dueDate"] timeIntervalSinceReferenceDate];
    int nowTimeInterval = (int) [[NSDate date] timeIntervalSinceReferenceDate];
    
    int clampedDueDate = dueDateTimeInterval - (dueDateTimeInterval % SECONDS_IN_A_DAY);
    int clampedNowTimeInterval = nowTimeInterval - (nowTimeInterval % SECONDS_IN_A_DAY);
    
    return (clampedDueDate - clampedNowTimeInterval) / SECONDS_IN_A_DAY;

Works for me for the cases I test. If I have made a silly mistake, I’m sure it will be made known shortly after this post hits the weblog.