Yeah, I can’t avoid this one.
Wil Shipley wrote a great article about his opinions of unit testing. Of course, the community response has been the fairly typical “He’s all wrong” / “He’s totally right” extremes.
Whatever. I would rather relay an anecdote.
As Duncan wrote, we used OCUnit to unit test Core Data. I integrated OCUnit into the Core Data project within a few weeks of the first lines of code being written. The experience was so positive that we decided to integrate OCUnit into Xcode (A huge note of thanks to the Sen:Te folks) and released the result at WWDC ’05 in Xcode 2.1.
Core Data 1.0 is not perfect, but it is a rock solid product that I’m damned proud of. The quality and performance achieved could not have been done without the use of unit testing. Furthermore, we were able to perform highly disruptive operations to the codebase very late in the development cycle. The end result was a vast increase in performance, a much cleaner code base, and rock solid release.
During the development cycle, we saw nearly zero regressions. And, of course, Core Data was developed on top of and along with Tiger. That is, we were frequently running in a highly unstable environment. Time and again, the Core Data unit testing caught problems that we are actually in components below Core Data.
Not only did the unit tests ensure the quality of our code, but it helped other teams to quickly find and fix bugs in other components.
Sure, writing the unit tests was occasionally tedious and maintaining the tests consumed a bunch of engineering time. I can say with complete and total confidence that the return on investment has been huge.
And the ROI continues to be high. When a developer files a bug against Core Data, the bug is generally turned into a unit test to verify that the bug really exists and the exact details. This greatly streamlines the whole bug fixing process and ensures that bugs stay fixed.
The team is also free to pursue some radical and disruptive changes to the framework for the next version with the confidence that it will remain completely backwards compatible.
Why such a different experience than Wil’s?
Software can generally be said to fall somewhere along a line between “infrastructure” and “end user application”. Core Data clearly falls at the “infrastructure” end of the scale while Wil’s application — the rather awesome Delicious Library — clearly falls at the other end of the scale.
And that clearly is the focus of Wil’s testing philosophy. Combined with his “the best software is the software with the fewest lines of clearly written code” attitude, his approach to testing is 100% entirely user driven.
That makes sense for his software.
I spoke with Wil at WWDC and he was very complimentary of the Core Data 1.0 release. I am thoroughly flattered by that as I have a huge amount of respect for Wil and that he was impressed by something I worked on makes me very happy indeed.
Given his “least code possible” philosophy, one could easily imagine that Delicious Library 2.0 or 3.0 will use Core Data as doing so would eliminate vast swaths of code he must have today. I hope he does exactly that at the same time he decides its OK to ship a Tiger+ only product.
And that is the answer. A software developer like Wil — a totally user focused, user experience driven, engineer — may not see the value of using unit testing directly in their development efforts. However, that Delicious Library could use something like Core Data and does use all of the other technologies in Tiger would not be possible without the huge investment in Unit Testing by the various teams at Apple.
Unit Testing is awesome for guaranteeing consistent behavior of infrastructure. UT can be very effectively applied to user interface level code, too, but it is a little harder to do.
But Unit Testing cannot test User Experience and, in the end, a quality UE is what turns a brilliantly engineered piece of software only appreciated by other engineers into a brilliantly designed products used by millions.