Where's Waldo?

End-to-end Arguments in System Design is one of my favorite papers in the computer science universe. It is well written, and it clearly states a design principle that was followed in the creation of the network we now know as the Internet. It gives some nice examples. What more could you want?

Well, I’d like more papers like that. In the computer science/software engineering/programming community, we don’t often talk about what makes a good design, or why we made the design decisions that we make. Oh, there are lots of book and articles that tell you how you ought to go about doing design. Maybe you should use pattern languages, or you should adopt an agile methodology, or you should do everything in UML diagrams. I’ve even seen combinations, saying that you should use an agile pattern language (no doubt producing UML diagrams). All of these can be useful as mechanisms for communication of a design, but I don’t find any of them actually talking about what makes a good design good, or how to go about doing such a design.

Writing about good design is far rarer. There is Butler Lampson’s classic Hints for Computer System Design. This is a great paper, but it is also pretty old (although not, surprisingly, outdated). There are a couple of books that tried to talk about design at different levels (Beautiful Code for programming, and Beautiful Architecture for collections of programs), but the results in both are mixed (full disclaimer; I wrote a chapter in the second and am not at all sure that it was much of a success). I’ve always like Paul Graham’s Hackers and Painters (the blog rather than the book), but it is more a contemplation on the art and craft of programming rather than on design. I’ve tried to talk about how to get good design, but it is a very slippery topic. Even Fred Brooks has written a book on the subject, which I like but is also, in the end, somewhat less than satisfying.

One of the reasons, I believe, for the lack of literature on the subject is that it is so hard to say anything that doesn’t seem to be either trite or just wrong. We all can agree that simplicity is a good thing, but what makes a design simple? Breaking a system into modules that make sense is trite; saying what it is for a module to make sense is difficult. You can recognize a good design when you see one, but explaining what a good design will be before the fact– well, I’ve never seen that done all that well.

After thinking about this for some time, I’ve come to the conclusion that good design is more art than science, more craft and taste than knowledge and process. As Brooks says, often the best that one can say is that good design comes from good designers. Good designers are often trained by having apprenticed with other good designers, but that’s about as far as you can go with the explanation. Even that training (which is really an apprenticeship) may not be all that conscious. I know of companies that set up mentoring programs in the hopes of getting good design training, but my apprenticeship (and I remember it well) was much more like that in Kill Bill, Vol. II– I spent a lot of time doing the computing equivalent of carrying water, and often felt bruised and battered. But I learned.

This is another way in which system design is like art– the way you learn to be an artist is to try, hear criticism from a  more experienced artist, and try again. Like a painting, the design of a system is never finished– there are always things you would like to change, decisions you would like to revisit. Sometimes you decide that it is good enough, and you ship. But you know you could always do better.