Everybody fears zombies. The only thing a product owner fears more than zombies, are zombie bugs.
Much like a zombie is a human being reanimated from the grave with a mindset for eating brains, zombie bugs are defects which were fixed, but “could come back”.
I have certainly encountered my share of “bugs which came back” in the early 2000s. It was a symptom (and indeed risk) of our Clearcase code configuration system.
Our project at the time MPA needed to do something radical. We needed to continue to deliver 3 monthly releases, but at the same time develop a whole new capability on a new branch for software that would not be released for almost 2 years.
The diagram below shows an example of the flow. Our starting code-base, version 1.0, was turned into two branches. Branch A had regular small releases, whilst branch B had lots of extra features added and developed for later merge.
The problem was that Branch A had a lot of features, but also some fixes of bugs that were in the original source material version 1.0. That meant those same bugs were also in Branch B, which wasn't receiving those fixes. This meant when we did our inevitable big merge down the line, there was a chance we could unwittingly take merge code from Branch B, which would reintroduce a bug we were sure we'd fixed. The zombie bug!
As per the norm for testers, this led to a fear of “the code regressing”, and caused a bit of an overreaction. Because bugs could “just come back”, automated regression suites were developed which checked for EVERY BUG EVER FIXED to ensure we'd not regressed.
Did I hear one of my audience go “ouch” out there? Yes indeed, before too long, our thorough testing had gone on to create more of a headache and backlog than the original problem. [If you're a fan of The Walking Dead, just think of all the pain The Governor caused. And he's not a zombie.]
These days, I'm not sure if I've just been on different projects that aren't as ambitious with their branching or if code configuration tools have got better, but I just don't see “zombie bugs” like I'd witnessed 10 years ago.
Certainly I realise the strategy was flawed. If a major merge is happening, the solution is to make sure you have plenty of time to give it a thorough testing, because it will be buggy. But focusing on seeking out “zombie bugs” means it's time you're not spending looking for new ones. And lets face it, in any build that I ever saw zombie bugs, I also saw a whole heap of brand new, non-zombie ones.
Is it really dead, or is it pining?
Hunting all out for zombie bugs these days is a waste of time. Do you really want to check that every typo you've ever found is still corrected?
Occasionally there is a big bug, a Godzilla bug, whose impact has everyone in the project a little scared (perhaps it was found in production, and caused customer issues). In this instance, to ease fears, it's probably a good idea to check once in a while that this bug has gone, so you can report once and for all “it is no more. It has ceased to be. It's expired and gone to meet it's maker. It's a stiff. Bereft of life, it rests in peace”.
Sometimes you have to say this several times to get the message across (it does help to have the evidence to back you up though). It's important business owners feels this has been put down once and for all … y'know, a double tap to the head.
So what's to be done?
Do I just play along?
The first thing to understand with code, the bigger the change, the more potential for introducing bugs. If you are dealing with a major merge, or a huge piece of functionality added, expect problems. But don't just go looking for issues where there have been problems in the past.
If you absolutely have to check for “bug fixes which have regressed”, make sure you are checking for the real nasty impact ones. Don't waste time hunting out trivial ones, which in all likelihood have little real impact.
But most of all try to make sure that this wild goose chase doesn't mean you're so obsessed with old bugs, you're failing to find new ones. After all, it would be a shame that if whilst focusing on an impending zombie apocalypse that never happens, you failed to notice that you were about to be overrun by cowboys riding dinosaurs ...
No comments:
Post a Comment