The Smart Way to Make Mistakes

Developers get better by making them, but by also learning how to correct and keep them from recurring. Here's a simple, three-step plan.

• By Mike Gunderloy
• 09/01/2006

There are, I think, only two kinds of developers who don’t make mistakes. At one end of the spectrum, I am willing to allow for the possibility of the super-genius developer who does everything right the first time, every time. This hypothetical developer doesn’t make any mistakes because all of their work is perfect, and they just crank out the code like a machine, day after day. Just because I’ve never actually met such a person does not, I suppose, disprove their existence.

Sadly, I have met the second kind of mistake-proof developer: the ones who never make a mistake because they never take any chances. If you’re only writing five lines of code a day, and they’re pretty much the same five lines of code you wrote yesterday, and you never learn anything new -- well, you might have a mistake-free universe. But you’re also not going to be all that valuable to the company. If your code output could be replaced by a photocopier, watch out! It’s time to learn the smart way to make mistakes. This is a skill that I try to teach to all of my direct reports, and one that has served me well over the years.

The Three-Step Plan
Let’s face it: At the end of the day, software development is hard. We’re out there turning pure thought into value, using a variety of creaky and imperfect tools. We’re constantly being pushed by bosses and clients to do more spectacular things, in less time, for less money. Mistakes are inevitable in this environment. You might make a tiny mistake, like forgetting to test a one-line code fix before checking it in, so that the code in the repository doesn’t compile. Or you might make a huge mistake, like accidentally deleting two million records from the production database, at a time when the nightly backups aren’t working. (Yes, as a matter of fact, I have done both of those things.) But it turns out that there’s a universal, three-step plan for dealing with these things that will make you a more valuable employee and a better developer:

• Claim it.
• Correct it.
• Counteract it.

Claim It
I don’t care what you just did wrong: Take a deep breath and admit that the mistake was yours. Don’t look around for a scapegoat. Don’t try to blame some underling for feeding you the wrong information. Don’t try to blame your boss for not buying you better tools. If you messed up, get over the human impulse to hide behind a bush and take responsibility for your actions.

Depending on the magnitude and visibility of the mistake, claiming it can require a variety of actions. It might just mean muttering something under your breath and going back to typing. On the other hand, it might mean dropping everything to craft an e-mail to your customer explaining why the site redesign is going to be delayed. Or you may have to pick up the phone and make some calls. Whatever you need to do, do it and accept the consequences -- because surely the consequences are going to be a whole lot worse if you try to hide things and get found out later. If the mistake has such horrendous consequences that an emergency plan of action has to be crafted, then you’d better get the relevant people working on it as soon as possible. Your job has a better chance of surviving that way.

Most mistakes, of course, don’t call for such extreme action. Claiming your mistake usually amounts to nothing more than admitting it to yourself, or perhaps sending an instant message to your boss or coworker saying, "Hey, I just did a really stupid thing." But this is still an essential, first step in the process of making mistakes the smart way. Until you defuse the tension that comes from being an idiot (and deep down inside, we all think we’re idiots when we make mistakes), it’s hard to get on with the rest of the program.

Correct It
With your adrenalin level back to something resembling normal, the next step should be obvious: Fix the problem. Depending on your mistake, of course, the effort involved in this step can range from trivial to impossible. If you managed to break the build, for example, it’s time to sit down, step through the source code, find the bug and check in a fix. But if you deleted those two million rows of data without backups -- well, it’s good to have an understanding boss.

But even if things can’t be put back together perfectly, this step is one where smart developers stand out from the average ones. If you’ve made a drastic mistake that just can’t be fixed, ask yourself: What can be done to minimize the consequences? Take that missing data, for example. Is it time to place a reassuring notice on the Web site telling customers that there was a server problem, but that their privacy remains intact? Is it possible to reconstruct at least the most important bits by looking at reporting summaries? Do you need to whip up a SQL batch to insert reasonable default records in place of the deleted rows?

Here’s the important question: When your boss asks "What are you doing about this problem?", you do not want to get that deer-in-the-headlights look. The best possible answer is "I’ve already fixed it." The second best is "I’m in the process of fixing it and will be done by such-and-such a time." Third best is "Here’s the plan for minimizing the damage." Anything beyond that, repeated too often, and you should be thinking about what size cardboard box it’s going to take to hold the contents of your desk.

Counteract It
Fixing mistakes, or minimizing their effects, is one of the things that I look for from those on my team. But the real key to smart mistake-handling is this: What are you going to do to prevent the same mistake from happening in the future? I tell my team members that I have a very high tolerance for making mistakes, so long as each mistake is treated as an opportunity to learn something that makes us a stronger, better team in the long run.

Suppose you broke the build. OK, so that’s not the end of the world. But if you make a habit of this behavior ... well, such are the things that unemployment is made of. You can generally avoid this problem by getting in the habit of updating your local copy of the source code from the repository and checking to make sure it compiles before checking in changes. A trivial habit, to be sure, but once you internalize it you’ve got a way to avoid making an entire class of mistakes. Problem solved.

Web server crashed and no one noticed for two hours while it returned an error message to potential customers? It’s time to recommend some monitoring software. Deleted a bunch of data from the production server because you were testing a new stored procedure that wasn’t fully debugged? You can avoid this by working on a development server instead and synchronizing changes only after they’ve been fully tested. You get the idea. I tolerate a lot of mistakes -- but please, never the same mistake twice! Surprise me with novelty, admit that you goofed, clean things up as rapidly as possible, and then make the whole process better. Iterate and we all learn.

Why Is This Hard?
What I don’t really understand is why this process comes as a revelation to people. Well, maybe I can. My conclusion, having had to lead junior developers through learning this approach more than once, is that too many organizations surround mistakes with a culture of blame. If admitting a mistake is immediately met with screaming, yelling, and loss of coffee breaks, then it’s very difficult to get on with fixing the problem -- let alone learning from it. The end result is a dysfunctional team where mistakes get hidden and finger-pointing gets elevated to a fine art, and learning opportunities are routinely missed. I suppose I shouldn’t complain about this too much; the fact that so many organizations are like this means that the exceptions can shine. I’m just glad I work for one of the exceptions. Hopefully you do too.

Want to read more of Mike’s work? Visit his Larkware site for daily updates at www.larkware.com.