Archive for the 'Mac OS X' Category

A while ago, I wrote about synchronizing iTunes libraries between a couple of machines using rsync while allowing for multiple locations.

I continue to use that system and it works great. Since I wrote that, iTunes has added significantly improved cover art handling. In fact, the whole cover flow thing has totally changed the way I interact with my library. Much more visual now. Has that feel of flipping through a stack of CDs to discover that hidden gem I haven’t listened to in a while.

With over 16,000 songs across several thousand CDs worth of music, I have a huge number of albums that don’t have cover art and don’t automatically resolve art from the music store. For automatic downloading of cover art, I have been using SonicSwap Boink. For the 30% of the time it mismatches cover art, I do a Google Images search to find the cover art.

Now, I have been using this command (paths modified, obviously) to backup my music library, including all metadata:

rsync -a -v --progress --block-size=220000 mastermachine:/Volumes/Music/ /Volumes/Music/

And I noticed that the synchronization process was taking a really long time recently! Much longer than would be expected given that I haven’t been ripping the rest of the CD collection recently.

It turns out that any track with updated cover art was having at least 220,000 bytes copied between machines. Given that the actual cover art is typically around 20,000 bytes and I couldn’t imagine an additional 200k of ID3 tags associated with the cover art, that order of magnitude difference in bytes changed vs. bytes transferred is a hell of a penalty!

The problem is the block-size. It is both a scanning window size for rsync and appears to also specify the minimum # of bytes transferred to describe a change in a large file.

I cranked it down to 15000…

rsync -a -v --progress --block-size=15000 mastermachine:/Volumes/Music/ /Volumes/Music/

…and the backups now go much, much faster. Of course, it’ll slow down the transfer of newly added music, but not by that much.

Update #1:

Denis suggested that turning on compression would fix the issue. To be specific, he suggested turning on compression in SSH and using rsync over SSH.

It only helps for the transmission of the iTunes database file(s). The media files, including the cover art, doesn’t compress by any noticeable amount when transmitted.

Pushing compression down from rsync into ssh may have the additional advantage of also compressing the rsync adminstrative noise, but larger block sizes are vastly more efficient when pushing media around with rsync!

So, no, compression won’t really help this problem for anything but the iTunes database files. Given their size — many many times the size of the average audio media file — I have been using compression and simply eating the slight CPU overhead related to fruitlessly compressing the already compressed media.

Ridiculous Fish (Peter Ammon) has released Hex Fiend as open source (truly open — BSD style license).

Hex Fiend is an absolutely brilliant hex editor. Beyond being a brilliant hex editor, it is a really really well implemented piece of software. Damned fast. Scales beyond the capabilities of the machine. Works really well and behaves consistently.

Anyone who claims Cocoa is slow needs to use Hex Fiend; it screams.

Even if you have no need of a Hex Editor, I highly recommend checking out the source for the learning experience.

svn co http://ridiculousfish.com/svn/HexFiend/trunk/ hexfiend

There is also an awesome Wiki describing the implementation of HexFiend.

Update: This is kind of like the recent switch in basketballs used in the NBA. Go read Mark Cuban’s discussion of the new ball. Very interesting. Some players benefit immediately. Many do not.

Erik J. Barzeski asks For those TextMate users in the audience, why do you use TextMate? For those BBEdit users, why have you stuck with BBEdit?

Text editing with any great speed is a tremendously intimate bit of interaction between you and the editing software. It is inherently a keyboard driven task and a very complex task, at that, given the myriad of syntax rules and gestural editing possibilities.

As a result, any user who spends any significant amount of time with any one text editor — I’m talking years, here — will build up a set of usage patterns that employed rapidly and repeatedly throughout an editing session. Often this is called “muscle memory”, but it is really more that your brain builds up a library of “mental macros” that are applied almost subconsciously as you work with the editor.

Because of this, switching text editors is incredibly disruptive to one’s workflow and results in some awesome “religious wars”. Why? Because it is just too damned difficult to actually quantify why one editor is so much better than another.

I have written at length about this before.

So, really, it boils down to how many of the “mental macros” can be transported from one editor to another.

Me? I was an emacs user. Because Cocoa’s text editing subsystem has supported emacs like behaviors since 1990 or so, the move from emacs to TextMate was relatively natural. I lost a lot of the more complex features, but I gained a much more modern and native user interface.

An acceptable trade off.

I have tried BBEdit many times and simply can’t use it. It makes my skin crawl to try and do so. Certainly not because it is a bad piece of software or otherwise inferior, but entirely because my learned mental model of text editing is simply incompatible with BBEdit’s implementation.

As I said in the above article:

Using BBEdit literally makes me grind my teeth with the general discomfort it causes. Now, I know that’ll rub a few people wrong (if anyone reads this) so let me be completely clear: BBEdit is a brilliant product and my inability to deal with it is entirely in my head.

So, in this case, it sounds like Erik’s brain has remapped itself to view text editing within the patterns of use perpetuated by BBEdit. As a result, a move to TextMate or SubEthaEdit is going to be disconcerting. It will feel unnatural and the flaws/bugs are going to be amplified.

A couple of days ago, I blogged about the release of Aperture 1.5 and how I really dig the refinements and performance boost. Using the photos at right as a [macro -- hey, I'm a happy flower photographer and I'm not afraid to admit it] example, I was able to use the color tools to refine a photo in ways that I couldn’t before.

I’m not the only one digging the new stuff. Duncan — a professional photographer from which I have learned a huge amount — takes on the Aperture color controls and also offers a comparison with similar features in Adobe’s lightroom Adobe Photoshop Lightroom.

Neat stuff. Now I need to go and adjust a few photos given what I just learned…

One of the spiffy new features of Aperture 1.5 is the ability to drag-n-drop images out of the Aperture window and into anything that can accept a JPEG.

When I first did the import, it was one of the first things I tried and it didn’t work. I was bummed. But then I realized that it was tied to the preview image generation.

If a photo hasn’t had the preview image generated yet, it can’t be drag and dropped. As soon as the preview generation process is done, everything works as expected — drag and drop works beautifully.

BTW: If you haven’t done so, check out the profiles provided on the Aperture site. The Vince Laforet profile is quite interesting; he describes how he took the shot of the guy messing with the light bulb on top of the gargoyle head about a zillion feet up the side of the Chrysler building in NYC. Pretty neat setup.

Now, I just need to find me a dentist’s office that is going out of business…

Aperture 1.5 shipped on friday. Brilliant update. Wow.

I live on a laptop, a MacBook Pro. As such, my hard drive is a bit constrained. Prior to 1.5, I had moved my Aperture library off to an external hard drive. A bit of a pain to have to carry around and plug in the external drive just to use Aperture.

With 1.5 I can move about 90% of the photos– the master images– in my library off to an external drive, keeping only the most recently taken and most frequently accessed on the internal library. Sweet.

Beyond library management, Aperture adds several new image processing tools that are simply amazing (bearing in mind that “amazing” equates to “even a clueless user like me produced something that I found pleasing with relatively little confusion”).

For example, the two images to the left are a sort of before/after 1.5. The top image takes advantage of the new color adjustment tools and the edge sharpening tools. I took the original photo to try and capture the cell-like nature of the bubbles and I think the new-and-improved image does a much better job of doing exactly that.

This is another example of using the individual channel color adjustment tool to improve a photo. In particular, I was able to adjust the colors that were specific to the background to decrease the distraction of the gunk that was behind the flower when I took the shot.

Beyond the new tools, the existing tools within Aperture have been improved. For example, the loupe is far more powerful than before beyond also not getting in the way nearly as much. I really like how I can lock the loupe to one location and then drag it to a new location. It makes fine grained adjustments, especially spot/patch/red-eye editing much easier and more precise.

Neat trick: With the loupe visible and set to “focus on loupe” mode, stick the cursor over some interesting spot in the photo and hold down the ‘`’ (show/hide loupe key). The loupe will slide to that new location. I end up using this all the time as I make adjustments or view images.

I have no idea if it has been improved, but the “spot” and “patch” tools feel much more responsive and powerful in the 1.5 release. Previously, dealing with spot/patch was just too damned tedious to play with the adjustments enough to actually get a decent result.

The Flickr integration is totally brilliant, too. Thank you, Fraser, for a brilliant bit of engineering, there.

Seriously, the whole release feels faster and I think it is making some of the tools feel more powerful even if they are otherwise unchanged. I am nowhere near a professional photography. I really have little clue what I’m doing beyond futzing around to discover things I like and don’t like. It really feels like Aperture 1.5 is letting me explore more possibilities and better refine my photos.

I have pretty much given up on iTunes gracefully handling a media library that spans multiple hard drives. I really like the “Keep iTunes music folder organized” as it makes it much easier to go traipsing through the library. As well, I use “Consolidate Library…” all the time. The latter truly insisting that all media live under one folder — on one device.

Since hard drives are ultra cheap, it really isn’t a problem right now. My library is just under 200GB and I have it living on a 500GB My Book. Finally, I have scanned nearly my entire CD collection (224kbit AAC — once drives are cheap enough, I’ll go for lossless).

Of course, backups pretty much require backing up to another hard drive (or drives — I would rather have multiple copies of this much data, just in case). Now, wouldn’t it be nice if said backup were also usable from within iTunes on another machine?

Easy enough.

(1) Relocate your music library and metadata to an external hard drive. This is a combination of first moving the metadata out of ~/Music/iTunes*, launching iTunes with the OPTION key held down, and then using the advanced menu item to point to the new music location.

(2) Backup the library with rsync:

Assuming you are on the machine that will act as the backup, do the following every now and then:

rsync -a -v --progress --block-size=128000 mastermachine:/Volumes/Music/ /Volumes/Music/

At this point, you’ll now have a complete copy of the music and metadata from your master machine on the local machine’s “Music” volume.

If you do exactly what I have done, even the paths will be the same. You merely have to launch iTunes with the OPTION key held down to re-home iTunes to the metadata found /Volumes/Music/.

If the volume name is different or you are rsync’ing to a sub-path, you’ll need to also set the advanced preference to point to the top level music directory. You can tell if you have set it correctly by whether or not the music can actually be played.

Now, the iPod part…

I discovered that if I connect an iPod to either machine, iTunes does not warn that the iPod is connected to another iTunes library. The iPod is quite at home on either machine.

The end result is that I can now treat my master machine as just that — the master. All metadata updates, etc, are sync’d to that machine. Whenever I rsync, the metadata is propagated out to the backup machine(s). I can still plug the iPod into any machine and it will sync down new music based on my neat little set of smart playlists.

If I produce additional metadata on the iPod in the form of song ratings (and, of course, “last played data” that I might care about), I simply need to remember to sync it to the master machine.

Of course, this all just accidentally works. So far, it works well enough. I’m not worried about data loss as everything between the iPod and iTunes seems to be rather transactional in nature. That is, there isn’t some kind of a bulk binary database load from the iPod into iTunes. If the iPod is hoarked, it is just a matter of restoring it to the factory settings with the updater.

Update: Alert! Language Police Action in the Comments! (Which, btw, I totally dig. Thanks!)

Subversion 1.4 was released recently.

Among a whole bunch of bug fixes and enhancements comes a significant new feature for Mac OS X users.

Subversion 1.4 now supports the Mac OS X Keychain. But don’t go looking for useful documentation or information in the release notes. There isn’t any. It was covered by ticket #2339 and committed as revision 17619. svn log -r17619 http://svn.collab.net/repos/svn reveals the details of the change.

Still, no information on how to use it. And that is because it “just works”. So, how to “upgrade” my existing authentication information to use the keychain (instead of storing the passwords in clear text in ~/.subversion/auth)?

Quite easy, but not automatic. If you remember all of your repository passwords, simply delete the ~/.subversion/auth/svn.simple directory.

Svn will recreate it and populate it as necessary.

Me? I can’t remember my damned passwords, so I moved the directory aside for reference and am deleting each entry from it as I re-authenticate with 1.4.

To verify if a repository is using the keychain, have a look in the appropriate server’s file within the ~/.subversion/auth/svn.simple directory. It will be named something like dd6654cecce24382249525337a2b0a4a (grep is helpful about right now).

For servers using the keychain, you’ll see something like this:

K 8
passtype
V 8
keychain
K 15
svn:realmstring
... more stuff deleted ...

For servers using the old-school, plaintext password in an unencrypted text file, you’ll see:

K 8
username
V 4
bobdobbs
K 8
password
V 8
DoYouReallyThinkI'mThatDumb?
... more stuff deleted ...

John Gruber has discovered that window contents remain live during the genie minimization effect on Mac OS X. There are other animations on the system that will also be slowed by the shift key. Exposé is the most obvious.

Here is a stupid/fun little hack:

• Type killall Dock into a Terminal window (but don’t hit return)
• Minimize a window while holding shift (slow minimize)
• quickly bring forward the Terminal window and hit return

The Dock controls the minimization animation and, when killed while it is running, the above will leave the warped window on screen in its warped form. All of the contents in the window continue to display and work correctly, though mouse tracking is completely borked.

(I don’t remember how I stumbled on this one.)

Hex Fiend 1.1 is out. Lots of tweaks, fixes, and polish.

… and also one very funny and educational write-up of how to maximize read throughput when scanning really bloody large files.

I don’t need a hex editor often. When I do, Hex Fiend just completely kicks ass. Fast. Flexible. Just works.