Sunday, May 13, 2012

To ISTBQ or not to ISTBQ, that is the question …

It is without doubt one of the most contentious points in software testing at the moment. “Do we need an official certification to be software testers”.

There are arguments all over the place, and I've found myself inspired by an article by my fellow Twitter collaborator Jari Laakso to write up my own opinion.  You can read his article here.

In favour of certification of course is the fact that “anyone can call themselves a software tester”. Any industry where you can just call yourself a role without needing some form of registration, means that the “cowboys” bring down the reputation of the industry. I might be able to call myself a plumber, but the almighty mess I'd leave your house in, coupled with a crippling bill for call-out would lead you to curse plumber-dom if I was allowed to behave in such a way.

However against certification, there is the fact that it very much becomes a whole “industry” in itself to sell you the course, the exam and the shining certificate. And how do you measure someone's ability? Frankly the way it's examined in a multiple choice exam is laughable – give any good tester options A, B, or C and they will always come up with a brand new answer D.

I myself qualifying for the ISEB exam back in 2005 – it was a compulsary course for my company, and I took it with someone who was new to testing. I found myself wrestling with the syllabus, because in many ways the material hooks into such an idealised form of software delivery that I've never seen it in the real world. I kept finding myself asking “but what if ...” and “but surely ...”.

That said, I really enjoyed the course, and I managed to learn from some formal ways and background theory of doing things which I'd previously just done intuitively (without really being able to say way). And despite being the slight heckler in the class, I formed a good relationship with Rob my tutor, and we kept in touch for a few years afterwards.

As a test manager myself I do find myself looking for someone with the foundation qualification, as I know we'll have a relatively similar framework of testing vocabulary to work from.

However I also agree with a lot of the criticism. The exam based on this certification is based on multiple choice and a rigid syllabus. It's the idea that there's a “right way” and a “wrong way” to test, like everything is black-and-white.

Sure there are many “wrong ways” to test. But there is also no single “right way” to do it either. Testing is taking the pure theory of software delivery model, and then using imagination to find a best model for the project in front of you, the way it demands updates, the way software is developed, the methods used to update your test environment. You don't put together a test plan using multi-choice …

I had a great boss (Stephen Pedrick) in 2005 when I took my course. Not only did he book me on the ISEB course, but he booked me on a following course “Testing – Putting Theory Into Practice”. This dealt with really looking at the theory and making practical test plans, evaluating sample projects. This was almost an antidote to the ISEB course, taking the theory we'd learned (which I did enjoy) and really thrashing it in real work scenarios.

The ISEB course was good, the Putting Theory Into Practice was invaluable. Sadly though, where 30 people took the ISEB certification, only 4 took the one follow on (for which you didn't get a shiny certificate). And this indeed is one of the many potential traps of certification, believing that certification is “all you need” to train as a tester.

It's not, as Obi Wan said to Luke Skywalker, “You've just taken your first step into a larger world” ...


  1. Here are my thoughts on ISTQB:

  2. I should add here as well, Elisabeth Hendrickson started Entaggle as another way of "recognising" your peers and people you work with over using the certification route to prove your a competent tester ...

  3. To hire anyone that calls themselves a Tester it is proportional risky as hiring a ISTQB certified Tester. What if one is certified but lazy or slow worker or focused on doing the easy stuff or doesn't like Testing? What if he was certified many years ago and never did testing at all?
    Maybe "is candidate certified" is not a great metric. I would try multiple other metrics as: "does candidate have good observing skills", "is he an easy learner", "is he a team player", "does he like testing", "will Testing be a temporary job for him", "can he think out of the box", "can he explain what he did when testing and why", "can he prioritize bugs" and my favorite "can he find important bugs fast".

    1. "What if one is certified but lazy or slow worker or focused on doing the easy stuff or doesn't like Testing?"

      Absolutely right - this is where we'd rely on a reference. And you're right, if someone gives me a good reference for my testing work, it's that more of a comment on my character than a piece of paper!

  4. The question "Do we need an official certification to be software testers?" has long been settled. The answer is "No". There are many employers who will hire you without having an official certification. Thus, you don't need one.

    We should stop beating dead horses. Pretending as if this were still an open question is an injustice to our profession.

    Now if you want to argue "Would the world be a better place if every employee in every profession was required to have some sort of official certification", or "Are testers better off having an official certification?", or "Would testers on my test team be better off if I got them all officially certified?", or "Would I be serving my company well by hiring only testers who had an official certification?", then there might be room for discussion.

    I'd most likely still argue on the side of "these certifications are not worthwhile", but at least we might be discussion something more relevant.

    1. Unfortunately in New Zealand, we're a bit slow, and I am indeed still having this conversation with a lot of companies who require testers.

      Testing is a form of service, and it's sometimes very hard to talk a customer (someone wanting testers) out of a conception they might have.

  5. Part of me says "what harm can it do", but then I thought more about it.

    There's a test manager where I work who is utterly hopeless. I believe he has ISEB certification, so I can only assume he used it to lend credibility to the smoke and mirrors that got him through interview.

    Given that I wouldn't trust him to test if a light switch works there is no way that having an ISEB accrediation in any way correlates with testing skills.

  6. Sorry, I thought this was an open discussion of the subject. I guess not. If you're only interested in comments you censor then I guess I'll go share my views with others who are interested in debate ratehr than pretending to be interested in debate

    1. I don't censor anyone's opinion IT Pro - I'm sorry if you've lost a comment somewhere, but I've not recieved it.

      I know Google are making some changes behind the scenes to the blogspot, which is making it damn difficult for me to do a few things myself -> I cannot currently edit my ex-posts, which is annoying as I've misspelt someone's name - which just seems rude on my part!

    2. It definitely appeared after I posted it, then it disappeared after maybe 30 minutes - let's put it down to a technical glitch then

      As I recall the thrust of what I posted was:

      Initially my thought was "what harm can it do".

      Thinking about it some more however, there is a tester where I work who is (at risk of sounding like I'm exaggerating) the worst tester I have ever come across. I understand he is ISEB certified, so I can only assume that qualification was used as part of the smoke and mirrors that got him through interview.

      Since I wouldn't trust him to test a light switch I have to assume that (for ISEB at least) there is no requirement to be a capable tester to get the qualification.

      Unfortunately in that case it's doing more harm than good, as I have to assume the manager who hired him took the qualification at face value in lieu of getting someone who can question a tester involved in the interview process.

  7. I enjoyed the debate around certification, but would like to have spent some more time on the issue of test apprenticeship. I had an “old-school” test apprenticeship that included being part of the Y2K “hysteria” and went through a kind of testing boot camp through a local IT recruitment agency, that bonded me to the company for 2 years and got me my first testing gig w a government department. From them on, it was “native cunning” and learning off “older hands” as I moved onto other jobs, in bigger teams.

    While I agree that there is an element of fear involved in acquiring a certification (ISEB, ISTQB) to make you look more marketable, I am a firm believer in experience being the best endorsement (proof) of skill. You need to get burnt a couple of times to learn how to do things better next time.

    Sitting the ISTQB Foundation exam, but more importantly for me – reading the supporting book, validated that what I had learnt myself was common to others’ experience as well. It gave a name to some of the techniques I use. Later, attending the Test Manager course, lead by Brian Osman, further galvanised my belief that my own experiences were valuable, as rang true w the other course attendees i.e. we all had war stories that sounded similar. (The text book says to do it this way, but in the real world ...).

    As was attested to on the workshop – testers asked their managers to put them through certification, to give them credibility in the market, w their peers in the other disciplines (BA, Dev, PM, etc), and w the client as well. I was one of those managers. I don’t feel I did my team a disservice, by putting them through certification. For some of them, certification was the push they needed towards further enquiry/study. Speaking from the position of a trained teacher – there are different learning styles. Some folk are disciplined/confident enough to strike out on their own, and make a good fist of doing so. Others need to be lead.

    As test leaders, we bear a certain amount of responsibility towards educating our peers/direct reports. I’m not saying that certification is the only answer, and I’m certainly not pro-lining someone-else’s pockets, but I also don’t want to limit someone by closing a door either.