Improving Testing is Not Safe (A parable)

Assume you cut your hand in the kitchen, then didn’t notice for an hour and had to be rushed to the ER. You do this 3 times per day.

You spend most of your time in the ER and very little time cooking. You want to fix that.

The improve testing strategy says “we’re going to cut ourselves all the time. How could we notice that so that we don’t bleed out?”

So we write something that will check every minute to see whether we’ve cut our hand in any of the specific places we’ve cut ourselves before. Of course, if we cut ourselves in the leg, we still don’t notice it and we end up in the ER.

And then we start checking our leg. Over time we are catching many our cuts in time to put on a bandage. We spend less time in the ER, but we’re still all covered in bandages and spend a lot of time looking for wounds.

Safeguarding would instead ask “why are we cutting ourselves 3 times a day? Why is this kitchen so unsafe?”

So we look and realize that the tilted counters are causing knives to slide off, people lunge for them so that the knife isn’t ruined on the floor, and then they cut themselves.

Safeguarding is to flatten the counter top.

Treat bugs as fires

A couple centuries ago, fires were seen as a natural side effect of cities. If you put that many businesses and houses together, sometimes it would just all burn down. Cities were useful, so you accepted that every once in a while a whole lot of people would die in a fire.

Much of the software world is in the same state today. Bugs are seen as a natural side effect of code. If you build systems that solve that many interesting problems for people, sometimes it will just all screw up. Software is useful, so you accept that every once in a while a whole lot of people’s work will die in a bug.

The people building and debugging software today have similar methods of working to the people building and de-firing cities then. But today, fire fighting and city planning are different. They prevent fires. Fires still happen, but no cities burn down and few people die. Let’s do the same with bugs. Continue reading “Treat bugs as fires”