Schrodinger's Code ...

Recently Elisabeth Hendrickson published an excellent article about Schrodinger's Cat and software.  The infamous Schrodinger's Cat is a famous thought experiment (by the physicist of the same name) used in quantum mechanics.  The cat is placed inside a sealed environment, with a poison gas capsule which has a 50% chance of being released each hour.

It's then covered it up, you walk away and return to it in an hour.  At this point the cat can be in one of two states – alive or dead.  But you cannot be sure for certain either way, it's 50:50.  The only way to be sure is to lift the cover and take a peek.

I will repeat again, it is a thought experiment – meaning you think about this, and not do it (that's animal rights off my back).  But it's more than a thought experiment, it's an emotional experiment too.  People like cats in general, with only the odd proto-serial killer taking exception to them.  So although even in the thought experiment you know from the logic that there is a 50% chance the cat is dead, emotionally you want and hope the cat to still be alive.  [Well maybe YOU shouldn't have put it in the death trap to begin with]

But I see the same phenomenon which I call Schrodingers Code on waterfall projects, especially outsourced waterfall projects …

Project managers will send work out to development teams, and the work will start to run late.  But that's okay because the developers are promising that everything will be done one week before release is needed.  The project is running over it's plan, but it can still be delivered.

As a tester you'll ask to get in and test early versions of what development is putting together.  But your project manager insists that would only get in the way and it would be better to leave the developers to “get on with it”.

Can you see what's happening there?  Someone has put a cover on the death trap, and even though four hours have passed when it should have been left for just one hour, they are desperate that the cat should still be alive.  Because they really need it to still be alive.  It then becomes project management by desperation.

What happens then is that the code is delivered, but under simple tests it falls over.  Under questioning, the developers admit “we were so busy getting it to you we didn't have time to test it”.  The problem is that what needs fixing means the software is far from finished and there's a lot more still to do.  The Project Manager and Business Owner conclude “everything was going well until testing got involved”.  The tester sighs and looks for a new job …

It's one of the things which have most struck me recently – what I call the death of unit testing.  Having moved not only from the United Kingdom to New Zealand but also from military software to commercial applications, it's stark and sometimes shocking on past projects.  The shortness of timescales, the ambition to deliver more functionality in less time.  Usually it's testing which can all too often end up compressed.

Outsourced companies are desperate to deliver something to the project plan, even if they know it's not tested, so they can claim they've met their contractual obligations.  And everyone hopes that a single phase of systems testing can cover a lack of testing anywhere else.  [It could happen is the reasoning]

No wonder there's a growing underground army of testers saying “it's not done until it's tested”, because until it's tested we have no idea if it's any good.

I'm a big believer in testing often and early.  I did a presentation to some developers about testing, where as an ex-developer I spoke that the greatest reward you can have is the first time you run some code you've been working for weeks on, and experience for yourself the thrill of it working.  It's something developers should be not just encouraged to do, but find the true joy in.  It's like someone making toys which they never get to play with.

What am I asking in terms of Schrodinger's Cat?  Well basically that we lose the cover, we keep an eye on the cat the whole time, and if it gets into trouble, maybe we need to rethink it all, smash the glass and rescue the cat, rather than hoping the cat gets lucky …

The fundamentals of testing ...

I've been working to try and capture the fundamental aspects of testing, and what our our "prime directives" and place within projects should be.

Here's a revised version of what I feel are the essential drive and vision for a testing department.  Whatever your product, whoever your customer is, you should find yourself nodding along to the majority of this.


The value of testing

Testing is important. It holds the key to safeguarding the quality and reputation of services from your organisation.

Projects most testers will encounter during their career will be diverse in their ambition, scale and partnerships. Thus a one size-fits-all approach will not work, but there are some essential traits which testers should identify as necessary for their role.

Mission statement


Testers should be dedicated to serving and championing its customers and business users. To that end the greatest customer experience is quality and trust from our customers in the services we provide.

Testing in your organisation should be dedicated to ensuring that any product you deliver will, wherever possible, continue to support your customers trust in your organisation.

Your aim is to understand, document and assess any risks within a new system, and aim to prevent any surprises when that system goes into production.


The Principles of Testing


The following principles should be upheld and championed by testers,

Objectives

• The aim of testing is to ensure products are fit-for-purpose and to enable them to be delivered.

Assisting Project Management
Testers should,

• Use their experience within the software industry to mentor Project Management on the testing process, as well as seek to support the decisions of management.
• Regularly report to project management regarding progress of testing activities and expected timelines.

Defects
Testers should,

• Work to illuminate high impact defects rather than report cosmetic defects en mass.
• Work to verbally communicate directly to development whenever this is possible.
• Communicate any high impact defect project management ASAP after it is raised. A similar attitude should be taken to any holdup which is delaying testing activity, such as delays receiving documentation or test environment not being available.
• Work with development and software providers, not against them. Testing is be an exercise done in partnership with these people to ensure the highest quality of product possible.
• Clearly communicate problems within defects.  Defects should include any reproducible steps, together with reference to requirements/design where possible.

Testing activities

• Early access to any software under development will help both scripting and early feedback on software.
• To best understand a product and it’s intent, software testing will need access to business analysts, technical architects and developers as well as any relevant requirement and design documents which allow understanding of the planned product.
• Where behaviour of a planned product is complex, testers should take leadership and ensure design walkthroughs occur.  Here the functional behaviour will be talked through with business analysts, architects, developers to ensure a working knowledge of the proposed product.
• The scope of testing should be scaled according to it’s appropriateness for this project rather than because it was done on another project.
• Test scripting should add value, and function as a repository for future regression testing.
• Wherever possible, testers should attempt to verbalise any issues encountered before formally communicating them.

Doing things better
Things don’t always go well on a project. Testers should be committed to learning from past mistakes to avoid repeating them. But neither should they overreact.





Deliverables
The following deliverables should be provided by testers for any test project,

• Test strategy document – initial scope and estimates for each testing phase applicable to project defined early on in the project, from which test budget can be calculated.
• Detailed test plan – based on test strategy, this is a more comprehensive plan for a particular phase based on available business/technical requirements
• Test scripts – a structured detail of what is planned to be tested / was tested on a software product
• Proof-of-testing – details of what was tested when and by who. Might be test script with tester name and date on.
• Defect report/notification – communication of any defect raised
• Test Exit Report – final document detailing what has been tested, what problems were encountered, what was fixed, what defects are outstanding, and why it’s considered acceptable.

Standard Risks
The following are standard risks which can affect testing timelines and budget,

• Late delivery of software for testing – the software for testing does not become available until later than planned, and hence testing is ready to begin, but cannot commence.
• Dependency on prior testing – within any test plan there tends to be dependencies that some testing has occurred beforehand. Otherwise testing will be the first point at which many defects are detected, impacting with a higher than expected level of retesting. Can be mitigated a little by asking for release notes of fixed defects, and test exit reports from prior phases which illustrate which issues were encountered and resolved.
• Time erosion – has any events which take testers off-task such as meetings been incorporated into the test plan. Meetings are a vital forum for understanding and communicating issues on the project, but the level of meetings needs to be known, and made sure to be “just enough”.
• Requirements – is there a single source of truth for technical, business and design requirements to cross reference? Are the people who wrote these documents available? This will help reduce unnecessary defects being raised.
• Build turnaround – the amount of time it takes development to turn around a new build and address defects is vital. Most software, even essentially good software, requires 2-3 builds to get polished for release. If your developers can only agree to a build weekly, and you have 3 days retest planned, this will not work. There need to be planned software release drops.
• Test environment – has the designed architecture been suitably understood that the correct test environment has been ordered and set up?
• Mis-estimation of scope – has an element or system not been assumed to be part of the test effort, but actually is? This will cause more to need testing than expected. This is why a Detailed Test Plan needs to revise initial estimates, and re-review in line with signed off Business and Technical requirements when they become available.

Let 2012 be the death of Superman!

Christmas break is a wonderful time for reflection on the year that's been, heck that's the reason we often find ourselves starting the new year with resolutions.

New Year's resolutions come in for a lot of stick.  But really the driving force between many good ones is a retrospective of the year.  I'm organising in January a meeting with our team as a reflection on 2011.  I think it's best summed up with “we achieved a lot of things in 2011 we can be proud of, but the idea of 2012 just being a repeat of the same ways of doing things is enough to make me cry”.  And yes, those were my words!

If I had to sum up something for myself which has to die, it is the myth of Superman.  In a nutshell it's the idea that anything is possible if all we put into it is a superheroic effort.  If we fail, we're not working hard enough.  It feels like something that harks back to the 80s mentality of “successful people make their own success”.

In actual fact as we all know the truth is often more that if we try to do more, we just burn out, get tired, end up making more and more mistakes, and actually deliver less.

Most of us who are professional seek to have a desire to “go the extra mile” (so 80s in itself), to try and move heaven and earth to make the impossible or ambitious happen.  We want to be the superhero who says “I can save you” to our project leaders.  This in itself isn't a bad thing.  We've all worked with naysayers and jobsworths who when presented with a problem will just shake their head, and cluck their tongue like a plumber who's about to present a really big bill.  These kind of people can be awful to work with and can be defeatist before you've begun.

There's nothing wrong with wanting to take on something ambitious.  We should feel stretched, challenged and engaged by our work.  As testers we are problem solvers, and we rise to a challenge.  Nothing is worse than just testing-by-numbers repeating scripts and formulas which were set down years before you - it's boring, and you feel undermined as a tester because you're not really doing anything creative or bringing something new to your work (PS – why haven't you automated such a project).

However the difficulty with taking on a challenge is to know when you're over your head, and having the courage to admit it.  This year as I've said I had a period of two months where I took a lot on at work.  And rather than do a few things well, I did a lot of things kind of poorly, struggling to keep my head above water.

The irony was although my managers were giving me these tasks, it was me who was putting myself under pressure to accept everything and try and "get on with it".  Superheroes like Superman don't admit defeat, even in the face of Kryptonian multitasking which is robbing us of our ability to do anything well.  They just try and plough on and still play the hero.

I've read two things this week which has reinforced a rather important message to me ...

Firstly was the mission statement for the 2008 season of training at Farnborough Rugby Club (I discovered whilst tidying up my office).  It's a wonderful document, stirring and inspiring (there are few corporate ones which manage that).  But within it is talks about building success through each player “looking to themselves to see where they need to develop, and where they need help, and looking to coaches and fellow players to achieve that”.

The other was a short story of mine, “The Saga of The Stone”.  It's a short story of a true life incident which happened to me in 1991, where as a young student I'd gone out for a night walk on the Moors, and had an accident.  Although I got myself out of immediate danger, I ended up coming out of the Moors the wrong side (20 miles from home) and suffering from hypothermia.  I was too determined and proud, and it took a lot for me to just ask for help / rescue from a local Church vicar.  In the epilogue I summed it up “it's okay when you're lost to ask for help”.

I think the Rugby club probably said it best.  Players were asked to identify themselves where they needed help, so they could become better.  It wasn't about admitting weakness, as much as building future strength.

So that's my resolution.  No more Superman, just my mild-mannered alter ego.  When I feel swamped, I'm going to say “I need help”.

Could it be that simple?

Some amazing blog articles ...

Was it only yesterday I said I was aiming to be quiet?

I did a review of the articles I've read in the last couple of weeks - most are favourited on Twitter (so easy to look back).

Interestingly, reading isn't a silent activity.  And by that I don't mean I mumble when I read.  If you come across good ideas they challenge you, and you also want to champion.  And so here are the articles which most have made me think this last month ...

The Role of Testing

Let's start with Scott Barbers article about 10 things about testing which should die.  If there is just one recommendation of mine you should read, it's this one.  I don't agree with a few of them, but they are really some great points there for you to review as a tester.

The big challenge there is really, stop behaving like a spoiled child.  The idea that some testers relish their ability to disrupt delivery as a kind of power trip.  This was echoed in another article by Eliza F, Oi you tester, where she points out "being a mean tester doesn't make you a clever tester".

It's perhaps right, something about testing we need to challenge, the trait inside us to look at a train wreck and go "if only they'd listened to us".  Rob Lambert also touched on this on the review of a conference, where there was a talk about "them and us" referring to "developers and testers".

I heartily agree, developers and testers need to have a working relationship.  Although my friend Russell went too far when he ended up marrying his tester, Maxine.  Like I've said many times, developers are like strikers in soccer, and testers are goalkeepers.  We do different things, but we're part of the same team, and we need to understand the ways we support each other.

In that vein then, I found the following presentation by David Evans about What testers and developers can learn from each other fascinating.

Talking about developers

If I have a second "must-read" recommendation it's the following article about developer habits.  We all know the stereotype of developers working until late.  Why programmers work at night is just a piece of absolute brilliance.

It explores why when dealing with something complex like programming, your mind needs to focus for long periods of time.  It's not something which can be stop-started by interruptions of meetings etc.  As an ex-developer I agree, when code has clicked in your head, you just can't get it written fast enough, and you need to be left to get it out of your head.  It's a fragile thing and interruptions disrupt it.

What I'd pick up from this is in an ideal world you'd try and have any meetings with developers within the first two hours of the day, and try and leave them to get on with it the rest of the day.  Fits in with the Agile idea of stand up meetings at the start of day.  Pester them and they're less productive.

I will give this to my project managers to have a read on Monday.

Bringing fun right back

Oliver Erlewin's article on Fun, IT and quality was interesting to read.  Are we losing sight of the fact testing should be fun, we should feel engaged rather than down-trod and under pressure?  I know I'm getting guilty of that.

Likewise I loved Brent M Jensen's discussion about how he socialised what testing was doing in his workplace in Speed date your way to better software.  These seem like quirky and silly exercises, but bringing fun into the workplace is about getting testers more engaged on what they're doing rather than doing the "test zombie" workforce who might as well be test automation machines, never varying what they test, never exploring.

Building on from that Olaf Lewitz discussed What makes a good tester, championing problem finders and problem solvers.  We're perhaps more famous as testers for the former over the latter.

And finally

Rosie Sherrie wrote a brief article on The tester and the marketeer.  I am working on more and more projects which are led by marketing, and I do hopes she revisits and expands this.


And finally - just for kicks.  We all know our favourite websites.  Everyone moans every time Facebook or Twitter changes or evolves.  This article reminds us what they looked like when they first launched.

Shhh - the sound of learning ...

I'm going to attempt to be very quiet this month, but I'm going to tell you why ...

Writing this blog is a great experience, and it really helps me in my work.  However as part of an experiment, I'm aiming this month to divert my energies from writing to reading.

Why?  Well just as a way of exposing myself to other testers ideas.

I'd been thinking about this anyway - but then was inspired by Lisa Crispin's article on Professional Growth in the December edition of Teatime With Testers (well there's also an article there by yours truly).

In it she talks of the importance to find time each week for some professional growth.  This is kind of true.  Most of what I've learned is self-taught through home study.

These days though I don't have the time to work through text books like I used to.  I might tackle an area or a chapter.

I find Twitter really useful, and keep a professional Twitter account.  I have a 25 minute commute to work via train, and use it to follow several testers around the world.  I find better than test books is reading blog articles.  They expose you to a small area, with a couple of ideas, and a little bit of opinion. But it's easily digestible.  And they often provoke ideas or reactions.

I'm also a big believer that the best way to learn about testing is to not just learn about testing.  Most of this blog is about looking at things like sport or films or Star Wars (not that Star Wars isn't a film) and looking for the parallels and the parables which we can bring back to our professional life and be better at our job.  Learn but in a fun way that it doesn't feel like learning.

Okay - so here are some great places to start.  There are a few online testers magazines out there, which contain an absolute rogue's gallery (but in a good way) of articles,

Testing Planet (which I sometimes edit articles for, but ironically never been published in) ...
http://www.thetestingplanet.com/

Tea Time With Testers
http://www.teatimewithtesters.com/

Testing Circus (which I've had the Agile Haka published in)
http://testingcircus.com/default.aspx




I'll try and return at some point and list some of the better articles I've experienced ...

A year of blogging and for what?

I'm coming up to a year of blogging  on testing, and it's interesting to reflect back.

I first came up with the idea of this blog really as a place to collect some useful materials.  I've always worked in my companies as a mentor, and imagined this place as a mentoring hub for anywhere I'd work in future, rather than my hard work being locked into the company intranet of somewhere I'd long left.

But as 2011 turned, I moved out of just testing and into test management in my new role.  My new company offered a lot of challenges.  Sometimes I'm frustrated that “we're not there yet”, but we're making progress, all-be-it slower than I'd hoped.

Work has presented me with a few frustrating moments and dead ends.  I've found it useful from tough weeks to sit in front of my machine, and talk about what's gone wrong – but try and stay positive, and talk about how to make things right.

I think an article I personally go back to a lot, is the Kobayashi Maru of Office Relationships.  It was a very tense time at work, and somehow talking about it and breaking it down helped.  I've worked really hard since to build bridges with Pauline, and I'm hoping we have a much better relationship.  Ironically through the bridge building process I've found out just how much pressure she was in, and she's admitted to not always handling it that well.  What could have become a bitter office feud is turning slowly into two mates trying to watch each other's backs – relationships take some building.

Another stand out article I've written has been The Agile Haka, about getting teams to work better, which got published in The Testing Circus, and my General Manager read this month and gave it his seal of approval.  And finally Those Darn Test Estimates, which is more identifying when testing overruns – I refer to those root causes regularly with my Project Managers now.

Those were my favourites, though ironically not the most popular of my posts.  You can never second-guess your readers.

I thought I was going to write this blog to teach.  In fact I found every time I sat down to write, I actually learned.  Just sitting down and thinking of an aspect of work I was unhappy with helped me look at the problem, break it apart, and look for answers.

What else did you expect?  I am a tester after all ...

Why is it so important to be able to socialise after work?

We had a great night last week after work. A departmental meal and karaoke night had been organised for the Friday.  Everyone talked with each other, and often with someone new during the meal. We enjoyed a few drinks, and we got really into the spirit of things becoming quite competitive on the microphone.

It's important to like the people you work with (although I'm usually better friends with people outside of work). But it's important to do these events once in a while because there's an important though subliminal message within these things as long as it's all fun and games.

The office is a place of tension. Project managers want one thing, business owners another, testers and developers something even more different. I discussed a little about office tensions in the Kobayashi Maru ofOffice Relationships – I've worked many different projects in 15 years in the IT business, and it's a similar story everywhere.

But in a social environment, when you get on and even have fun with others, there's an important thing you come away with. No matter what the strains of the 9-til-5, you can get on with each other. Yes it's going to be tense at times and sometimes emotional. But it's nothing personal.  And that's something important to come away with.