Category Archives: Code

Dangerously Confusing Interfaces

confused.jpgDesigning intuitive interfaces that are easy to use and easy to learn is hard, often very hard; and for economic reasons it might not always be possible to strive for perfection. Nevertheless, in my view, at the very least, interfaces should be designed such that obvious, day-to-day usage doesn’t lead to damage.

In his classic book “Writing Solid Code”, Steve Maguire calls confusing interfaces that lead to unexpected bugs “Candy Machine Interfaces”. He tells a story from a vending machine at Microsoft that used to cause him grief: The machine displayed “45 cent” for “number 21”, but after he had finally inserted the last coin he would sometimes enter “45” instead of “21” (and would get a jalapeño flavored bubble-gum instead of the peanut butter cookie that he wanted so much — Ha Ha Ha!). He suggests an easy fix: replace the numeric keypad with a letter keypad and no confusion between money and items would be possible anymore.

The other day I did something like this:

My goal was to recursively copy the ‘gamma’ folder to my home folder. What I expected was a ‘gamma’ folder within my home directory, but instead I ended up with hundreds of files from the ‘gamma’ directory right at the top-level of my home directory — the ‘gamma’ directory simply wasn’t created!

I have to confess that similar things sometimes happen to me with other recursive-copy-like tools, too — this seems to be my candy machine problem. Now you know it.

As for ‘rsync’, there is a feature that allows you to copy just the contents of a directory, without creating the directory, flat into a target directory. Granted, this is sometimes useful, but do you know how to activate this mode? By appending a trailing slash to the source directory! That’s what happened in my case. But I didn’t even add the slash myself: if you use Bash’s TAB completion (like I did) a trailing slash is automatically appended for directories…

But good old ‘cp’ puzzles me even more. If you use it like this

it will copy ‘from3’ to a folder named ‘to2’ under ‘to1’ such that both directories (‘from3’ and ‘to2’) will have the same contents, which is more or less a copy-and-rename-at-the-same-time operation. Unless ‘to2’ already exists, in which case ‘from3’ will be copied in ‘to2’ resulting in ‘to1/to2/from3’. Unless, as an exception within an exception, there is already a ‘from3’ directory under ‘to2’; in this case ‘cp’ will copy ‘from3’ flat into the existing ‘to2/from3’ which might overwrite existing files in that folder.

Both, ‘cp’ and ‘rsync’ suffer from fancy interfaces that try to add smart features — which is normally good — but they do it in an implicit, hard-to-guess, hard-to-remember way — which is always bad. Flat copies are sometimes useful but they might be dangerous as they could inadvertently overwrite existing files or at least deluge a target directory. A potential cure could be an explicit ‘–flat’ command-line option.

To me, a wonderfully simple approach is the one taken by Subversion: checkouts are always flat and I’ve never had any problems with it:

This copies (actually checks-out) the contents of the ‘trunk’ flat into the specified destination directory — always, without any exceptions. That’s the only thing you have to learn and remember. There are no trailing backslashes or any other implicit rules. It will also create the target parent directories up to any level, if needed.

Naturally, dangerously confusing interfaces exist in programming interfaces, too. Sometimes the behavior of a method depends on some global state, sometimes it is easy to confuse parameters. The ‘memset’ function from the C standard library is a classic example:

Does this put 40 times the value of 32 in ‘buffer’ or is it the other way around?

I have no idea how many programmers don’t know the answer to this question or how many bugs can be attributed to this bad interface, but I suspect that in both cases the answer must be “way too many”. I don’t want to guess or look up the specification in a manual — I want the compiler to tell me if I’m wrong. Here is an alternative implementation:

Now you write

If you confuse the fill character with the length parameter the compiler will bark at you — a parameter mix-up is impossible. Even though this is more to type than the original (dangerous) interface: it is usually worth the while if there are two parameters of the same (or convertible) type next to each other.

Like I said in the beginning: designing intuitive interfaces is hard but spending extra effort to avoid errors for the most typical cases is usually a worthwhile investment: don’t make people think, make it difficult for them to do wrong things — even if it sometimes means a little bit more typing.

The Principle of Proximity

pebbles.jpgIt is common wisdom that opposites attract. In programming, however, it is desirable to keep things that are related together — that’s at least what the “Principle of Proximity” states.

This principle has many manifestations, some of which are well known by most software developers, for instance:

-Keep the documentation (comments) as close as possible to the code;
-Initialize variables as close as possible to the point where you use them;
-Limit the scope of declarations (i. e. use namespaces and don’t make constants public if private is sufficient);

As opposed to opposites, related things not always attract, or — as a matter of fact — attract in a suboptimal way.

Here is an example. Assume that you have to process a list of different objects (let’s call them “boxes”, for the sake of this example) that you have just received, maybe over a socket connection. This list always consists of a blue box, a red box, and a green box, exactly in that order. These boxes are encrypted and protected by an integrity checksum. Before actually processing them, you need to perform decryption and integrity checking. (Also assume that the boxes are completely different. They have different content, different security mechanisms, and require different processing.) Below is one way to go about it:

At first glance, this code doesn’t look bad at all. It is grouped in such a way that the three steps are clearly visible: 1. get a box; 2. apply security to box; 3. process box. If you zoom out a little, the structure looks like this:

Is this the principle of proximity in action? Are related things close together?

Not really. The things that are close together are the objects under each operation, but the objects themselves have little in common. Contrast this with this approach:

The structure is now inverted:

The objects and their operations are close together; in fact, they are completely encapsulated. I like to call this ‘encapsulation at runtime’, which is not to be confused with traditional object-oriented encapsulation where you put data and its related operations close together at coding time, in a class. (Which is another instance of the principle of proximity, BTW.)

What I don’t like about onReceiveBoxes1 is that it mixes up things that are unrelated: order of boxes and order of box actions. Just because the boxes are ordered in a particular way, doesn’t mean that we have to perform the box actions in that particular box-order. Unnecessary dependencies are usually bad for maintenance.

Ah, maintainability, that’s where the second implementation really shines! If you have to add a yellow box someday, you just copy and paste the block of an existing box and do some minor modifications. And if the order in which boxes arrive changes, adapting onReceiveBoxes2 is likewise trivial. Better maintainability means that the risk of introducing an error is much lower, which in turn means that you spend less time debugging and have more time for doing code katas.

Honoring the principle of proximity almost always gives you better efficiency, either. Notice that in the first implementation, the pointers to all boxes have a fairly long lifetime and must be kept in memory (or CPU registers) as they are needed until operation 3 has finished. onReceiveBoxes2 only needs a pointer to the box that is currently worked on, which means that the compiler only needs to allocate one pointer.

A Small Matter of Style?

Some claim that there is a reason for everything. Well, I’m not so sure about this statement, but surely there is a reason for Code Kata 1: it was inspired by a routine that I found in a real-world product some time ago. It looked more or less like this:

The job of this routine is to find a value in a vector that matches the given reference value as close as possible. It works by visiting each element of the vector, calculating the absolute difference to the reference value and, if the new difference is smaller than the previous, updating the ‘bestIndex’, the index of the value that matches best.

This routine has proven to work reliably, at least no errors have been found so far. But in my view, it nevertheless leaves a lot to be desired. Let’s dissect this code step by step.

First of all, there are no comments, but I don’t think this is a big issue here. In my view, the code is quite short and the identifier names give sufficient meaning.

The most noticeable peculiarity about this code is the hideous floating point literal 1.7976931348623158e+308 on line 3. What the heck is this?

If you study the for loop and find out how the algorithm works it becomes clear: ‘bestDistance’ is initialized to the largest possible value such that the condition on line 8 becomes ‘true’ for the first element, in order to get a first candidate.

But using a hard coded floating-point literal is the worst approach to achieve this. Why not use ‘DBL_MAX’ (from cfloat/float.h) or ‘numeric_limits::max()’ (from limits)? Especially the former is much easier to type, less error-prone, and you get portability for free: recall that the limits of numeric types are not fixed in C++ (for double the upper bound is specified to be at least 1E+37, but that’s about all that the language standard promises).

The algorithm itself is not perfect either. Like I said, during the first iteration, the comparison on line 8 will always fail as ‘bestDistance’ is initialized to the largest possible value. Why not save one iteration by initializing ‘bestDistance’ to the distance between the reference value and the first vector element? Sounds like a good idea, but this approach will work only if we know that the vector is not empty. Which brings us to the topic of the odd way in which errors are handled in this routine.

Have a look ath lines 14 and 15. The routine returns the found ‘bestIndex’ only if it is within the bounds of the vector; otherwise it returns -1 (for failure). Under what circumstances is it possible that ‘bestIndex’ is outside the range of the vector? It turns out that it can’t — unless the vector is completely empty. So why not make this check explicit and put it near the top of the routine before the actual search starts?

Here is an improved implementation, based on my previous observations:

But I’m still not happy with this routine: as it stands it works for doubles only, but isn’t it of general utility? The code would be identical for vectors of chars, ints, pointers, and complex numbers. Why not turn this code into a template function such that it can be put in a library?

There is a snag, however. The code uses ‘fabs‘ a function that returns the absolute value of a floating point value. We need it to be able to compare the distances of vector elements to our given value, but it only works with floats/doubles and ‘abs’ is for ints/longs only, so we cannot write truly generic code. But wait a minute, doesn’t cmath provide overloaded versions of ‘abs‘ that work with all types?

So our generic implementation could look like this:

Looks neat, but we still have a problem: for simple scalar types, we can assume that ‘abs’ looks like this:

which means that the return type is the same as the argument type. Hence, code like

is perfectly fine.

But for complex types, like complex<>, the picture is different as complex is itself parameterized by a type:

and according to the mathematical definition, the return value of ‘abs’ on a complex number is not a complex number but rather a real number:

Therefore, if T is complex<double>, this line

doesn’t compile: ‘bestDistance’ is of type complex<double> but the return value of ‘abs’ is double.

Are we stuck in a rut? Sometimes the result of ‘abs’ is T (eg. for int, float), sometimes it is something totally different (eg. for complex, valarray, pointers). It looks like we cannot provide a truly generic implementation.

Not so fast! If we use C++0x and one of my favorite features, automatic type inference, we can:

The ‘auto’ keyword is magic: in C++0x a variable declared as ‘auto’ gets the type of the expression that is assigned to it — what a great tool for developing generic code!

There you have it. Even though the original code worked fine, the final version is nicer: it provides better readability, portability, and genericity. On top of that, it is theoretically slightly more efficient because it saves one redundant comparison. Why theoretically? Because I haven’t measured execution times. What a breach of style. Shame on me.

Killing Your Loved Ones

mouse_trap.jpgYou have been working on this new feature for days. Suddenly you are stuck. You are faced with a problem that you haven’t thought of so far. After some painful hours of blood, sweat, and tears you finally have a Eureka! moment: a brilliant idea that will solve your problem in a very elegant way, but it will take a few days to implement.

After you’re done, you feel overjoyed: what a nice solution, what a beautiful design (you even used template metaprogramming), well-crafted doxygen comments, no compiler warnings and — needless to say — all the unit tests pass. You walk around the office with a smile on your face, you want to tell everybody what marvels you have achieved… alas, the third person you bump into tells you that the problem you have solved is in fact no problem at all — neither was nor is. Perhaps in the distant future, but surely not today.

Your heart is pounding, you feel dizzy, you feel sick. You try to defend your work, but your colleague’s arguments are compelling.

What to do, you ask, after having gone through so much pain and effort; what to do with this beautiful code? I tell you what: admit defeat and take it out again, period.

We’ve all been there. Sometimes it’s just a few lines, sometimes many classes, worth many person-days: It’s not really required for the product, but it is already implemented. In such miserable moments, you can add the most value to your project by deleting what’s not needed. I know it is hard, due to what psychologists call cognitive dissonance, but it must be done. Remember Antoine de Saint-Exupéry’s words?

“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”

Usually, programmers have little difficulty accepting software minimalism principles as advocated by agile development practices like Extreme Programming (XP). As Ron Jeffries notes “Always implement things when you actually need them; never when you just foresee that you need them”. Clearly, every feature requires time to write, document, and test. And according to the YAGNI principle: “You Ain’t Gonna Need It”. But what if the time and money is already spent? Why not keep the code, it surely doesn’t harm?

The sad truth is that it does harm, even if it is already written, tested, and documented. The more code you have, the more complex a system becomes. Every additional line of code makes a system harder to read and understand, especially for new team members. It makes the system a little bit harder to refactor and maintain. Maybe someone wants to reuse your feature in another project, but rather reinvents the wheel because he doesn’t want to read and understand all that extra code. Further, such unnecessary code often leads to more unnecessary code in the same or other parts of the system.

Let me put it this way: “Anything that’s not worth implementing is not worth having in your code base”.

You needn’t (and shouldn’t) throw your code into a black hole, though. Push it on an experimental branch, or tar it up and put it in your project’s Wiki. This helps you to get over your loss and if you really need it someday, it’s there for you to be resurrected. Until that day comes, it’s best for your and your teammates if your code rests in peace.

The Beauty of Imperfection

leaves.jpgThere once was a master programmer who wrote unstructured programs. A novice programmer, seeking to imitate him, also began to write unstructured programs. When the novice asked the master to evaluate his progress, the master criticized him for writing unstructured programs, saying, “What is appropriate for the master is not appropriate for the novice. You must understand the Tao before transcending structure.” (from The Tao of Programming)

Some time ago, I worked on a little command-line tool, when all of a sudden I stopped. Did I really just use a ‘goto’ statement?

What surprised me most about this code was that I didn’t feel any sense of guilt; somehow, it was just natural to use a ‘goto’ in this context.

Isn’t it considered harmful to use ‘goto’ statements? Or global variables? Or public attributes instead of using accessor methods? Or to duplicate code and data structures? In general, I think it is (of course) but there are circumstances when it is perfectly OK to write such “imperfect” code: If you know what you are doing and if you sacrifice perfection for a higher-level goal. This is very similar to a gambit in chess-playing: you trade a piece for a strategic long-term advantage over your opponent.

In my case, the higher-level goal was to actually do write a little tool for my personal use that allowed me to do things automatically — things I used to do manually (and due to the pain of manual work, frequently omitted). Granted, it would not have taken much effort to put the error reporting code into a routine of its own. But I guess that if I had spent enough time thinking about all software quality aspects like readability, testability and documentation, I would have never written this otherwise useful tool. It’s always better to have a tool that saves you from manual labor but uses goto statements than no tool at all. So my use of a ‘goto’ is just one particular manifestation of my “go-as-fast-as-possible-or-it-will-never-be-done” mindset.

You could also call this style of programming “deliberate hacking”: you know that you do something suboptimal but you do it on purpose for good (sometimes noble) reasons. This style of hacking should not be confused with bad hacking, where you write bad code either unconsciously or consciously for a bad reason (like pure laziness or obscuring your code to make yourself indispensable).

It is a good idea to keep software quality gambits in the private implementation details, where as little people as possible are affected. As a C programmer, do you care about any hacks in the ‘make’ program’s source code? Most likely not! However, many ‘make’ users complain about the fact that ‘make’ is whitespace sensitive: you have to use TAB characters to indent build commands — spaces may result in obscure errors.

For sure, a software quality gambit increases technical debt, but it is important to note that you get an asset with a much higher value in return. Ideally, the value of your asset doesn’t decrease (or only slowly decreases) over time, so that the technical debt isn’t noticeable or at least doesn’t harm. I’m convinced that the decision to use TAB characters in ‘make’ was a good hack at the time, but its asset value has long become zero. One possibility would be to change ‘make’ such that it works with TABs and spaces, but is it really worth it? For instance, how confusing and annoying would it be if makefiles that use spaces (and work with the latest version of our imaginary ‘make’) result in obscure errors on systems where only older versions of ‘make’ are present? It’s probably the best to embrace this historical beauty mark as a given fact.

In general, I think we must fight perfectionism in software development to avoid losing track of our main goal: shipping working software with good enough quality. Next time you find a hack, try to understand its nature and motives. Was it a good hack or a bad hack (frequently good hacks are marked with explanatory comments, like TODO comments). Maybe you can even develop a sense for appreciating such good hacks. This is akin to “Wabi Sabi“, a Japanese art concept. Wabi Sabi is sometimes described as seeing beauty in things that are imperfect, impermanent and incomplete. Wabi Sabi art acknowledges three simple realities: nothing lasts, nothing is finished, and nothing is perfect. I think we should accept these realities for software as well.