Writing a kick-ass defect!

I was asked by a graduate tester about just what should go into a good defect report.  It's a really good question, and to my surprise, something I've never written about, although in a way whether we give defect reports in a written form, or a verbal form, thinking about the information you provide when reporting a defect part of the key role of a tester.  After all, if we're not able to tell someone else, that bug isn't going to go away any time soon!

Obviously, different project will have different defect templates.  However you're bound in your career to find places where you'll find none at all, so thinking about what you need to provide is a valuable skill.

I've written many defect reports, but I've also been a developer who's fixed them.  So I know what's helpful, and what's not.  Something I really encourage new testers to read is a humorous meme of supposed log book communications between pilots and ground crew.  The pilot is vague about the issues, so the ground crew are vague about their resolution back.  Don't let this happen to you!

As with most things - writing a great defect answers some very generic questions of What-How-Why-Where-When!

So let's start the ball rolling - you're using your companies new trial piece of software, and something just doesn't seem right about it.  Let's start defining it ...

What

In a nutshell, what is the problem?  This should be a real brief summary of what the problem is.

I like to think of it in terms of elevator pitches.  Imagine you've just got into the ground floor elevator with your project manager.  You get off at the 3rd floor, she gets off at the 4th.  And she asks you "hey, I heard you found a defect this morning?".

You have 3 floors, and about 20 seconds to summarise what you've encountered.  This is a real skill in summarising what you've found to a one sentence tag.  But believe me, defects like episodes of Friends are remembered as "the one with the...".

If you have a super and concise "what the problem is" summary, it probably belongs as your title.  That way, anyone reading through your defect system will go "ah".

Hint - when you go to talk to a developer about a defect you've logged, it's best not to quote just the number.  very few people remember bug 666.  That "the one where ..." summary will be the best way you have to job their memory!

How

Okay - you've summarised the problem.  But how did you cause it to happen?

The how is a way of "repeating your steps".  Ideally with a defect, you'll be able to repeat a series of steps, and repeat the unusual behaviour.

Repeatability of a defect is great.  But sometimes you just can't repeat the behaviour.  What then?  Well that's when you have to use your judgement - often depending on how much of an issue you think it was.  It's always worth talking through with a developer.

This is one of the reasons I like to use a screen recorder when I can, because it records exactly what I did and where the issues happened.  It is alright though to have defects that can't be repeated, but it's best to put on it "I'm not able to repeat".

Oh - and they say a picture is worth a thousand words, so never underestimate the power of a screenshot!

Why

Why is this a problem?  For you to be raising this as an issue you must find there's something you don't like about what you've seen.

Sometimes it's very black and white, "the requirement says the page should display your account details, and instead it displays nothing".  Likewise, if you encountered a blue screen of death, you can be pretty sure that's not "as per design".

But sometimes you might raise a defect because there is something going on which doesn't feel right.  It bugs you (another word we use for defect).

Your description of why will lead you to another description of a defect which is important - it's severity.  Typically in projects there are more defects and oddities than time to fix.  So people tend to focus on the things which are causing the most pain.

Severity is how severe a problem something is.  If you're causing your machine to blue screen regularly, that's pretty severe, and going to impact what you can test.  If you have a spelling mistake, that's less of a problem, and certainly not going to impact your testing too much.  However as it doesn't take much of a spelling mistake to make a swear word (as the test manager who emailed me with a mispelled request for "defect counts" found out).  And although it's true to say we testers live to report these kind of defects, they can actually have a functional impact - if you have a system which generates an automated email which includes a swear word, it's likely some e-mail filters are going to put it in the junk pile!

Generally though - although there are grids and standards for defect severity, I've found you just tend to pick this up through experience.  In truth, if you get everything else right about your defect and leave the severity blank, most people can choose appropriately from the information you've provided.  But experience helps you to find tune this.

Where and When

When it comes to retracing your steps where it happened and when it happened helps.

Where - well just that.  You might have a couple of test environments, so it helps to know that.  If you're on a web application, the kind of browser (and version) usually helps.  And indeed the machine.  All this information goes double for mobile devices of course!

When can be handy too - sometimes it's known for someone to be doing a release to an environment, and forget to tell testing.  Shocking huh - but it does happen.  Or indeed it allows a developer to look through the logs around about that time to see if anything peculiar was going on in the logs.