Mixing clever and stupid

The most valuable technologies I have built had at their core some pretty clever ideas. That wasn’t all of it, though. They also had some stupid ideas. Not stupid bad but stupid crude and stupid simple.

For instance, I worked on a load-balancing system with some ingenious methods for distributing load across a pool of servers. I also had a service registry. Kind of. They were text files with hostname:port pairs, one per line. The core idea was really clever. The service registry? Dumb as a sack of hammers. This is the sweet spot.

The easy case is something that is stupid top to bottom. If there are no clever bits, what is it doing for you? It might be doing something useful, but it’s probably not that useful. It’s probably inflexible and narrow, like a basic shell script.

Consider a project that is clever all the way through. Maybe the core of it is some insane gradient descent algorithm, and that should be clever. But what about its logging? What about how it takes input? What about how it allocates and frees memory? Do those need to be clever? Not only do they not need to be clever, but it’s almost certainly harmful if they are.

Every bit of cleverness adds implementation cost, maintenance cost, failure modes, and difficulty debugging. Software programs follow the Pareto Principle, but to an extreme. Instead of 20% of the code contributing 80% of the value, it’s probably 4% of the code contributing 96% of the value. The rest is just bookkeeping and logistics. You certainly want those things done well and efficiently, but they’re clear candidates for satisficing. Save your maximizing for the 4%. Making the 96% fancy won’t make your product any better, and it’ll almost certainly make it worse.

Leave a Reply

Your email address will not be published. Required fields are marked *