Why Developers Make Terrible Testers of Their Own Software — and What to Do About It

There is a peculiar problem with testing software you have written yourself: the more familiar you become with it, the less capable you may become of seeing what is wrong with it.

Developer Dan Luu describes this phenomenon as bug blindness. His argument is simple but uncomfortable: people are remarkably good at adapting to broken, awkward or unintuitive systems. Once we have learned how to compensate for a problem, we gradually stop noticing that the problem exists at all.

For software developers, this is particularly dangerous.

We know how the application is supposed to work. We know where things are. We know which buttons need to be pressed, which operations take a moment to complete, which sequences work reliably and which ones should be avoided.

Eventually those behaviours become automatic.

At that point, we are no longer testing the software an ordinary user will encounter. We are testing the software through a thick layer of accumulated knowledge and unconscious workarounds.

And that makes us surprisingly poor judges of our own work.

The Workaround Becomes the Workflow

One of Luu's software examples involves Google Docs.

Over time, he had learned that certain operations did not behave reliably if performed immediately. Rather than continually thinking of this as a bug, he simply learned to pause before performing the next action.

Eventually, waiting was no longer a workaround.

It was simply how he used Google Docs.

That distinction is important.

Imagine an application occasionally fails if you click Save immediately after changing pages. After using the software for several months, the developer instinctively waits half a second before clicking Save.

Ask that developer whether saving works and the answer will probably be:

“Yes.”

Technically, that may even be true.

But give the same software to somebody who has never used it before and they will change page, immediately click Save, and encounter the problem.

The developer doesn't encounter the bug because the developer has unconsciously trained themselves not to trigger it.

This happens everywhere.

We learn not to double-click something too quickly. We know that a dialog takes a moment to initialise. We know that a particular button doesn't make sense until another operation has completed. We know that changing a setting requires reopening a window before it takes effect.

None of those behaviours necessarily feel like workarounds once we have been doing them for long enough.

They simply become muscle memory.

That is the essence of bug blindness.

“It Works” Is Not the Same as “It Is Usable”

Developers naturally think about software in terms of implementation.

Given input X, does the program produce output Y?

If it does, the feature works.

Users approach the same problem from the opposite direction.

They know the outcome they want, but they do not necessarily know which input the developer expects.

A developer therefore tends to test this:

The real user experience is closer to this:

Those are very different tests.

This is why developers frequently describe interfaces as “obvious” that users find completely baffling.

They are obvious — once you understand the internal model of the software.

Unfortunately, users do not arrive with a copy of that model installed in their heads.

Dogfooding Helps — But It Isn't Enough

One commonly suggested solution is dogfooding: developers should use their own software.

This is unquestionably valuable.

A programmer who regularly uses the application is far more likely to encounter performance problems, workflow friction and edge cases than one who only runs it during development.

But dogfooding has an important weakness.

Developers adapt.

Use the application long enough and you will learn every shortcut, every strange behaviour and every workaround.

Ironically, this means that extensive experience with the software can eventually make you less representative of a normal user, not more.

The answer is not to stop dogfooding.

The answer is to change how you do it.

Instead of simply completing the task, occasionally pay attention to how you completed it.

Ask yourself:

• Did I just avoid doing something because I know it sometimes fails?
• Did I wait before pressing a button?
• Did I automatically retry something?
• Did I navigate somewhere in a strange way because I know the obvious route doesn't work?
• Did I ignore a message because I already know what it really means?
• Did I know where something was because I wrote it rather than because the UI made it clear?

Every one of those moments is potentially a defect hiding inside a habit.

A useful discipline is to mentally translate:

“That's just how you do it.”

into:

“Why do you have to do it that way?”

Very often, the answer exposes something worth fixing.

Watch Someone Who Doesn't Know the Software

The most effective antidote to developer blindness is remarkably simple:

Give the software to somebody who doesn't know how it works.

Give them a task rather than instructions.

For example:

Create a new document from this template, change something on page three, and print it.

Then watch.

And, crucially:

Do not help them.

This is much harder than it sounds.

Within minutes, they will probably hover over the wrong control, open the wrong menu or completely fail to notice the enormous button that has seemed painfully obvious to the development team for the last six months.

The temptation to intervene will be overwhelming.

“You just need to click…”

Don't.

That sentence is often the moment you discover the problem.

If the user cannot work out what to click without the developer sitting beside them, the interface has failed to communicate something.

Watching somebody use your software without assistance can reveal more in twenty minutes than hours of conventional functional testing.

Treat User Mistakes as Evidence

Developers sometimes categorise user behaviour as either correct or incorrect.

The user clicked the wrong thing. The user didn't read the message. The user misunderstood the option. The user entered something unexpected.

Sometimes that genuinely is user error.

But repeated user errors are data.

If ten people misunderstand the same control, the interesting question is no longer:

“Why don't they understand this?”

It is:

“Why does the software keep leading people to the same misunderstanding?”

The same principle applies to technical support.

If users repeatedly contact support asking how to perform the same operation, that should be treated as a form of usability telemetry.

A support question may actually be an undocumented UI defect.

The application technically supports the operation, but it has failed to explain how to reach it.

Test Workflows, Not Just Features

Developers commonly test features individually.

Open dialog: works.
Save operation: works.
Page navigation: works.
Print operation: works.

Unfortunately, users do not normally sit down and use one feature at a time.

They perform workflows.

They open a document, navigate around it, modify something, save it, print it and perhaps reopen it later.

Many problems only become visible at the boundaries between otherwise functional features.

For this reason, testing should include realistic scenarios rather than isolated controls.

Instead of:

Does the Print button work?

test:

Can somebody open yesterday's job, change the second page, save it under another name and print pages two through four?

The second test exercises both the software and the user's understanding of the software.

That is much closer to reality.

Occasionally Become a New User Again

Developers rarely experience their application in the same state as a genuine new customer.

Their machine contains configuration files, remembered paths, registry settings, cached data, recent-file lists and years of accumulated defaults.

Periodically testing from a genuinely clean environment can be revealing.

• Reset the settings.
• Clear the recent files.
• Remove the remembered folders.
• Use the default window layout.
• Start without the sample data you normally rely upon.

Then try to accomplish something.

This exposes another category of developer knowledge: information stored outside the software itself.

The developer remembers what needs configuring.

The customer doesn't know that configuration exists yet.

Encourage People to Point Out Irritation

One of Luu's more encouraging observations is that the ability to notice quality problems can be learned.

People who are repeatedly encouraged to identify awkward behaviour gradually become better at spotting it.

That suggests something useful for software teams.

Instead of merely asking testers:

“Does it work?”

ask:

“What annoyed you?”

Or:

“What felt unnecessary?”

Or:

“Was there anything you had to think about that shouldn't have required thought?”

Those questions produce very different feedback.

The goal isn't to turn every mild irritation into an urgent development task. Software always involves compromises.

But first the team has to notice the compromise exists.

Only then can it make an informed decision about whether fixing it is worthwhile.

Seek Critics, Not Confirmation

There is another subtle trap in software testing.

People who are familiar with a product often want it to succeed.

Developers, product managers and long-term users understand its history and why certain decisions were made.

That shared context makes them sympathetic.

Sometimes you need somebody without that sympathy.

Someone who will say:

“This makes absolutely no sense.”

Not because they are being difficult, but because they do not possess the contextual knowledge that everyone inside the project has accumulated.

That person is enormously valuable.

Good software teams should actively seek criticism rather than reassurance.

A tester who keeps confirming that everything works is less useful than somebody who continually finds situations where it doesn't.

Distinguish Technical Debt from Quality Blindness

There is nothing inherently wrong with knowing that part of a product is poor and deciding not to fix it.

Software engineering always involves trade-offs.

A team might legitimately conclude:

This workflow is awkward, but improving it would require three weeks of development and only a handful of users encounter it.

That is prioritisation.

The dangerous situation is when the team says:

This workflow is excellent.

while users are fighting it every day.

The difference is awareness.

Once a weakness has been identified, its cost can be compared with the cost of fixing it.

A problem nobody can see cannot even enter that calculation.

A Practical Anti-Blindness Checklist

Bug blindness cannot be eliminated completely. Familiarity with a system inevitably changes the way we perceive it.

But it can be deliberately counteracted.

None of these techniques requires an expensive usability laboratory.

Most require little more than another person and the discipline to keep quiet while they use the software.

Expertise Is Both an Asset and a Liability

A developer's knowledge of their application is obviously enormously valuable.

It is also precisely what makes objective testing difficult.

We know what the software means.

We know what it is trying to do.

We know where everything is.

We know its peculiarities.

The user knows none of those things.

So perhaps the most useful question a developer can ask when testing their own application is not:

“Does this work?”

but:

“Would this still make sense if I didn't already know how it worked?”

The difficulty, of course, is that you can never completely forget what you know.

Which is exactly why sometimes the best tester of your software is somebody who never knew it in the first place.

This article was inspired by Dan Luu's “Bug blindness”, which explores how familiarity with systems causes people to unconsciously adapt to defects and stop noticing them.