What exactly is the “design” of a software system? Is it a detailed UML diagram? Is it 300 pages of prose? I recently came across an article from 1992 by Jack W. Reeves, mentioned in an interview on the subject of TDD; in his article, Jack argues that the design is actually the source code itself.

What!? The code is the design? This proposition is surprising at first, but not really if you think about it: only the tested and debugged code is a precise description of how the system should behave, how it should be built. And the process of “building” a software system is merely the feeding of the code to tools like make, Ant, compilers and linkers. Thus, “designing” software is hard, building software is straightforward.

Often, people demand that the design should be a UML diagram, something that is easy to read and comprehend by anyone. However, for any non-trivial system, the diagram(s) would become at least as complicated as the real code. It’s always the same: you start out with a couple of classes that have only a few relations, and everything looks nice; but when you finally have 200 classes you suddenly cannot see the forest for the trees anymore. UML is good for highlighting certain parts and aspects of a system, but a complete specification, one that can be handed down to the “code monkeys” is a myth. Even the author of the most successful UML book ever, Martin Fowler, is very skeptical about UML as a blueprint design tool. As Fred Brooks pointed out long ago: “software is not inherently embedded in space and has no ready geometric representation” (like hardware or buildings do).

There are technical issues, too. How do multiple “designers” collaborate on a UML design? How do they create parallel versions and merge conflicts? How do they keep track of variants? Countless tools exist for designs based on plain-text (aka. source code); next to nothing exists for UML.

Why do so many folks fall prey for this “everything-has-to-be-designed-in-UML” idea? Some (most likely managers or sales people) simply confuse programming with typing. Others (most likely developers that never got the hang of programming), want to introduce a white-collar/blue-collar developer system. Yet others think that there must be a method (in this case UML) that would render a complex system in such a way that it can be understood fully by anyone (including managers, sales people and developers who never got the hang of programming).

The design of a modern airplane comprises hundreds of thousands of pages. Nobody can and must comprehend everything. But even laymen can look at the 3D model and understand a lot: ah, this is where the engines are, and yep, here we have the anti-lock brake system and, boy-oh-boy, here we have the flight controller.

Such design overview diagrams are very important for comprehension and communication and no source code can ever replace them. It’s important to have such a diagram (call it high-level design or system architecture diagram or whatever you like). If you want, you can draw it in UML but I suggest you don’t: UML is way too boring. Have a look at the Google Android architecture diagram — it’s shiny, it’s been reduced to the absolute minimum, it’s freakin’ awesome!

Is there a need for designers, anyway? I definitely think so! You need somebody who takes care of the big-picture technical decisions, like the high-level composition of a system, the coding conventions, the hardware decisions and so on. Let’s call this person the systems architect. And, of course, you need somebody who takes care of the look & feel, the user-interface, a Steve Jobs kind of person. And, last but not least, you need many small-to-medium scale “designers” who iteratively detail the “design” through so-called “programming” such that compilers, linkers and the like can build the rest easily.

It’s amazing how much time you can save by using regular expressions; it’s even more amazing how much time you can spend getting them to work correctly.

Because they are so powerful and easy to use, regexps can easily be misused, for instance by applying them to problems that are not “regular”, that is, where balancing is important:

Parsing problems like this are not suited for a regular expression matcher, as you need to retain state information and regexps simply cannot keep track of which blocks or braces are open or closed. In cases like this, what you really need is a parser. Period.

Alas, often people can’t be bothered writing a true parser, even if lex/yacc-like tools greatly simplify the work. And I’m guilty of this myself. Years ago I wrote a profiling tool for embedded systems. Since the embedded C code that I wanted to profile had to be instrumented (each function required enter/exit logging calls to get out the execution timing data) I needed to write a tool to do the job. I was not particularly interested in this job — hacking the actual performance analysis code was much more fun — so I decided, well, to go for a heuristic “parser” based on regexps.

In less than one hour I had cobbled together a little script that seemed to work fine. Over the next couple of months I had to spend endless hours fixing all the nasty corner cases; even today it doesn’t work in all circumstances! But I’ve learned my lessons: don’t use regexps when you need a true parser. Again, period.

But even if the problem is regular, people often define regexps sloppily. Look at the following example that checks if a .cfg file appears anywhere in a given string:

So let’s see what we’ve got here. We are obviously looking for a Windows-style absolute path: a single drive letter, followed by a colon and a backslash, followed by n optional directories (each of which followed by a backslash), followed by a mandatory filename that has a .cfg extension. Looks really neat…

These are the regexps people love to write and I don’t know how many times I’ve had to fix one because of this pathological “simplicity”. It might work today, but it is far from future-proof. Sooner or later the surrounding context will change and this regexp will match much more (or much less) than was intended.

Here a some of the major shortcomings:

– Using word characters \w is way too restrictive. According to the Windows long filename specification, a filename may contain any UTF-16 character, but for all practical purposes \w is really only a shortcut for [a-zA-Z0-9_]. If a filename contains a blank or umlaut, the expression won’t match anymore.

– Actually a corollary of the previous item: you cannot have partial relativity within an absolute path, e. g. C:\files\services\base\..\items\main.cfg would not match because the \w character class does not allow for dots.

– The regexp is not aligned on a word boundary, which means that if your editor happens to create backup files like C:\config\user.cfg~ they’ll match, too.

Often — but not always — using regexps means striking a careful balance between accuracy and convenience. It makes little sense to implement the complete Windows filename spec in a regexp. But investing a little energy to tighten them up usually pays off in spades. How about this?

At the cost of being only slightly more difficult to read, this solution is much more resilient to change due to the use of some good practices. First of all, it is easy do define a set of valid drive letters, so I used [a-zA-Z] instead of \w; second, the whole regexp is aligned on word boundaries, which means no more regexp over-/underruns and third, by stating that everything between separators (backslashes, in this case) is a series of non-separators we won’t run into “strange character” problems.

Next time you write a regexp think this: “I know that by using regexps I’m saving hours of development time, so I can afford to spend another 10 minutes to make them more robust”.