RIP Choppers

They've been saying that 2016 has been a bad year for celebrities - with a huge number of our UK childhood icons passing away.

This week has been more of the same, with the deaths of wrestling's Chyna, Britains much loved comedienne Victoria Wood, and iconic pop powerhouse Prince.

Then this appeared on my Facebook feed ...

Choppers, one of the last of the PG Tips chimps has died.

Thankfully - you might not know who the PG Tips chimps are - but in the 60s and 70s there was a series of adverts.  The ads involved chimps being dressed up to behave like human beings - it was a source of great amusement for many Britains.

Sadly though, we eventually found out there was a very dark side to our amusement.  The chimps lived in Twycross, and were a major draw there.  However one time my mother had a bit of a clash with the trainer.

It turned out the chimps were brought up and rigorously trained with human mannerisms.  The trainer who came across as an absolutely hateful woman was making a big deal about how the chimpanzees used for the adverts were carefully kept away from other chimps so that their behaviour didn't get contaminated by real chimps.

My mother - who is no Dian Fossey - was horrified, saying "not allowing an animal to behave naturally, never heard anything so ridiculous in all my life!".  Indeed it was the last time we ever visited.  Thankfully the ads fell out of favour in the 80s as we understood more about ape psychology through work of people like Dian Fossey.

Unfortunately the damage had been done - the PG Tips Chimps never really managed to integrate back with other chimps.  Indeed the current manager of Twycross Zoo is open about the damage that was done.

It's interesting to take time to think about this.  Zoos originally began as a Victorian curiosity where animals from around the world would be collected for the amusement of paying customers - with confined conditions and taunts from the public commonplace.

The PG Tips chimps are a relic of that kind of thinking, where the primary concern is the amusement of humans.  However zoos around the world have been trying to shake that image - now the focus is not so much amusement, but helping with the conservation of endangered animals, creating better habitats and considering animal psychology, and using the animals as ambassadors for why conservation matters.

It's a powerful reminder how to stay relevant, institutions need to be prepared to change, and to question the behavior and practices of the past.

RIP Choppers - and let's hope we've learned our lesson.

The danger of letting extremes and corner cases dictate our systems

I found myself quite sickened today reading through the news.  Anders Breivik recently won a human rights appeal case against how he is treated in his incarceration.

Breivik is a notorious Norwegian mass murderer who killed 77 people in several terrorist attacks.  Much like Beate Zschäpe, he identifies as a neo-Nazi - in only the way someone from Norway who was born after the Nazi occupation remotely could.

In my mind, the unrepentant killer is an absolute monster and there is no greater crime against human rights that the commitment of mass murder.  And so the verdict was somewhat sickening to me.

It's been the cause of a great amount of thought all day.  Stories like this often make the news, because we obsess how something in our society functions when we look at the "corner cases", the extremes of society we can't predict.

Breivik himself presents a conundrum - we was 32 when he committed his crime, but in Norway the law system has a maximum sentence of 21 years for any crime (although under the containment clause it can be extended).  They had never really imagined having to deal with a criminal like Breivik before.  Although murder can be found in any corner of the world, it was just still an unprecedented act in the country (ironically) since the Nazi occupation of World War Two.

We seem to be drawn to these kinds of corner cases in any of our systems,

• The mass murderer and his human rights
• The silly health and safety rules
• The social security fraudster

Often the obsession is not to fix the system - but to use these extremes to dictate the norm - which is usually used to support an overeaction that involves penalising the whole system.

We talk much about health and safety rules being silly (and some are interpreted trivially and obstructively), but lose track of the fact that they are in place to save lives in the workplace. And have helped to contribute to a fall in the number of work related fatalities by ensuring a right to safety equipment, and evaluating areas for potential problems.

The fraudster on social security is used as an excuse to make social security more restrictive and punitive.  When the system exists to help people in genuine dire straights (but they're often forgotten in the outraged hunt for the fraudsters).

In many ways, human rights as they stand in Europe are the legacy of Breivik's idol, Adolf Hitler.  After the monstrous acts committed by the Nazis, they are an attempt to legislate against ever seeing such state-fueled atrocities from ever happening again.

They are rights that are supposed to be there to protect each and every one of us.  The hideously bitter pill for me today was the realisation that for human rights to mean anything, they have to apply even to a monster like Breivik.

Today as I brooded on it, I would have loved to have a clause which could mean you could take away the human rights of someone like Breivik.  And then it had a ring of the familiar to it - and I realised that human rights can't be something we're entitled to.  They have to apply to all of us, the best and the worst, or they don't apply at all.

Once you create a clause that allows you to define someone as "not human", you have set a precedent.  You have created a reason to take away a persons human rights, someone else will find their own reason ... and another and another.  Once you have made it law that such rights can be ignored "in some cases", those "some cases" will always be exploited by others.  That's human nature.

Adolf Hitler managed to convince a country that the Jews were "sub-human" - much like I'm desperate do with Breivik.  Then it became the homosexuals, the mentally ill, and before too long anyone who disagreed with him.  We all know how that ended ...

Even in my old homecountry - you just need to look at the Prevention Of Terrorism Act 2005 which used the 7/7 Train Attacks as it's justification.

This act allowed special powers to opt our of human rights act - and would only be used to fight terror, right?  In fact it became one of the most abused laws on the books ...

• Police used to hold protesters/hecklers at political events - challenging the government is obviously the worst kind of terrorism
• Councils used it to spy on applicants for school places to determine if they really lived in a catchment area.
• As well as using it in order to catch people putting their rubbish out

It's a great example of the slippery slope that once you create the loophole, people will expand it to use it for anything.



In the end though, though Breivik won his case, it was a pyrrhic victory for him.  In supporting his human rights, we totally rejected his ideology - that we avoided taking a step down the path where we can ignore the rights of a human being, even one like him.

Somehow for our society, our systems, our thinking and our lives, we need to be able to cater for extremes and corner cases.  But not let the norm be dictated and overshadowed by them.

It's easy to forget, but when we protect the rights of someone like Breivik, we are making it that much harder to take them away from everybody else.

Thinking about test strategy ...

As background to this - this year I'm trying to collect a lot of ideas on test strategy.  It's my hope I might have enough material for a workshop on it next year.

Right now some of these ideas are getting structured, but at an early stage - so do contact me below or on Twitter with your ideas and comments.  I'd really like to push further on this!

Last week I was running an evening bootcamp on testing for Summer Of Tech.  It's an awesome opportunity not just to touch base with New Zealand's upcoming tech people, but also to champion a little bit about the fun and mindset of testing.

During the presentation though - I gave my current definition for testing ...

I've been using this slide for about a year - but realised how within that statement is something I'm thinking about more and more recentl.  Something that's been under our noses for a lot of the time.  Maybe we're so busy, blinded and programmed, because we've never really taken the time to explicitly think about it.

Control-Expect-Observe

Let's look at the standard test script format we've all had to use at one form or another ...

In a test script when we use action vs expectation we're really talking about "something I control" and "something I observe".

It's fair then to sum up a test scenario as using something we control to produce something we observe which can then be compared with something we expect.

All testing is a form of play between these factors - even exploratory testing which we talked about previously here.  We use a heuristic to vary something we control, we use an oracle to set our expectations, and we observe - raising defects if there's a variance.

Exploding the idea of strategy

At it's heart then, a test scenario seems a very simple beast - we control something, something happens ... is it what we expect?

This is probably where testing gets it's reputation for being easy - a single test scenario presented like that looks simple.  Even if we overlook how observation can be a difficult to master.*

The problem is that testing is not the execution of a single scenario.  The skill of testing is coming up with a series of scenarios which cover a system.  Again, people can come up with a scenario or two, but that doesn't make them skilled testers.

To me, being able to come up with a selection of scenarios to cover a product is not easy - and there are no guarantees.  There is such a vast variety of ways a product can go wrong, that it can be overwhelming.  Consequently we often will see members of the testing community talking about how huge testing can become.

Unfortunately that can become problematic.  I'm currently reading Daniel Kahneman's Thinking Fast And Slow, and we've come to the idea of substitution.

Simply put, when we're asked a very complex question, if it leaves us perplexed we will sometimes answer another related question for which we do have an answer.  In his book, when they asked students in one survey "are you happy?" followed by "how many dates have you been on this month?".  "Are you happy" is a complex question.  Put this way around, they found no correlation.

They then asked another group of students the same questions, but the other way around - "how many dates have you been on this month?" followed by "are you happy?".  This way around they found very strong correlation.  The reason being that when the complex question about happiness came up, they substituted a question they had just answered "how good is my dating life?".

Now remember how I mentioned how testers talk about "the possibilities of what you can test" being huge?  That means when a new system is put in front of us and we're asked "what will you test?", that it's a big complex question.

So no surprises, we tend to perform a substitution of our own, and hence we tend to answer the question initially as if it was one of the following questions ...

• "What do you control?"
• "How have you tested other systems?"
• "Where have you found problems before?"

Now none of those answers is a bad first answer.  The problem is when we're falling into the trap of using one of those without realising.  They are each a very good starting point, but if you don't push further, you'll end up doing shallow testing, and potentially missing something important.

"What do you control?"

I'm going to revisit material from a previous article on Back To Basics, where we looked at registration.

Let's take a look at Facebook registration for a change ...

It's easy to take a look at this and come up with a list of the following which are things we can vary and control ...

• First name field
• Surname field
• Mobile number field
• Re-enter mobile number
• New password
• Birthday drop down lists
• Female / Male radio buttons
• Create an account
• Various hyperlinks

When we look at these items our first approach is to create a list of test scenarios which use different data and permutations.  And that's as I've said is a superb start.  Where it's not great is if once you've exhausted ideas for this to consider you've tested everything.

Because you haven't.

What you've just done is a very competent job of functional testing.  You've chosen items that you know are simplest to control, and you've created scenarios.

What you've omitted are items for which you don't know how to control.  I think an important part of test strategy is being able to recognise "I ought to be able to vary X, but I don't know how" - then asking the question "how can I control X?".  Unless there's explicit instructions, we'll often have a blind spot for the "everything else" which defines a product which is often thrown into the "non-functional testing" basket.

What are some examples of tricky things you might want to control?

• Platform / Browser you test your system on.  Different browsers react differently - installing other browsers is easy.  And you can set up different virtual machines to simulate other platforms.  Maybe you need an Apple Mac (good luck prying it from your architect's cold, dead fingers) - and maybe some devices such as the latest iPhone or tablet?
• Screen resolution.  When was the last time you adjusted your screen resolution?
• Server load.  Does the system work well with multiple users on?  You can use a tool like JMeter to simulate a lot of users on your system - simply it's a tool that allows you to control a lot of concurrent calls to your system and measure response time.
• Hardware down.  If you've got a load balanced system, do you have a scenario with part of your system down?  It's not an obvious thing to control, but you should ask how you can get someone to do that.

Of course in talking about the above I've provided potential solutions - which was naughty of me.  You need in strategy to be brave enough to say "I need to be able to control X", even when you're not sure how.  Ask your team, and around anyway.  Maybe there's a tool out there, or maybe the team needs to build it into the system as part of testability.  But don't be too quick to say "I can't control X, so I won't bother".

At worst, if you can't find a way to control, then put it as an assumption "we weren't able to test around different X".

"How have you tested on other systems?"

Again a super method to get started, we use our experience on a previous system, and copy and paste it over.  This is great apart from,

1. Your testing will only ever be as good as your last project (what if instead of good testing, you were instead just very lucky?).
2. If the current system is almost identical to the last one, that's great.  But more than likely there will be differences - and do you need to account for them?

A good example, if you didn't need mobile device testing on your previous project, you're in danger of thinking that can apply to this project - are you sure?

"Where have you found problems before?"

Once again - this is a helpful method, with a hideous blind spot.  If you use only test areas where you imagine there could be problems, what if there are problems in the system you're just not imagining?  How can you think bigger in your testing approach?

If you only go looking for old bugs, the problem is you might not be taking a different path to find new ones!


A useful guide for strategy

Is there a particular approach you tend to favour?  In the early days, I tended to get stuck on "where have you found problems before".  I now tend to obsess a little on "how have you tested on other systems".

When you're looking at a new piece of work, capture all of your initial ideas and try not to think too much about it.  Once this has dried up, think how you've approach this - has it been as if you were answering,

• "What do you control?"
• "How have you tested other systems?"
• "Where have you found problems before?"

Think about the blind spots associated, and dig deeper.  Try answering the other two substitute questions, and see if it forces you to go deeper.  And of course, try out the Oblique Testing cards to shake things up a bit!






Regarding Observation

* If you look at my book How To Test, there is a chapter on observation.  However dispite over 400 readers, no-one has yet Tweeted me to say "where is Chapter 7?", this is one of the deliberate mistakes in the book -

Dan Billing has accused me of overplaying the "smart arse" card on this one ...

Take warning - Microsoft Office Assistant returns ...

I feel the need to take to the internet to warn you about ...

If you've applied a recent update then you'll have notice that the scourge of previous Microsoft Office version, Mr Clippy ...

Yes, the bane of all our lives is back!!!

But help is on hand - there is a way to turn him off if you're on an Edge or IE browser ...

Just look under Settings on your browser ...

Then you'll notice a new button - called Assistant ...

Select the option Deactivate Microsoft Assistant, then hit apply.

They hit the option Confirm, followed by I'm really, really sure.

And there - blessed silence.

Less than impressed at the ordeal Microsoft has turned this into.  I have some friends at the gym who work for Microsoft in New Zealand, and supposedly you'll only have a limited window to turn this feature off, before it's likely to become permanent - as a revenue stream to show you advertising through the assistant on your browser or any related software.

So get in there and turn the feature off before it's too late!