Saturday, April 23, 2016

RIP Choppers

They've been saying that 2016 has been a bad year for celebrities - with a huge number of our UK childhood icons passing away.

This week has been more of the same, with the deaths of wrestling's Chyna, Britains much loved comedienne Victoria Wood, and iconic pop powerhouse Prince.

Then this appeared on my Facebook feed ...

Choppers, one of the last of the PG Tips chimps has died.

Thankfully - you might not know who the PG Tips chimps are - but in the 60s and 70s there was a series of adverts.  The ads involved chimps being dressed up to behave like human beings - it was a source of great amusement for many Britains.

Sadly though, we eventually found out there was a very dark side to our amusement.  The chimps lived in Twycross, and were a major draw there.  However one time my mother had a bit of a clash with the trainer.

It turned out the chimps were brought up and rigorously trained with human mannerisms.  The trainer who came across as an absolutely hateful woman was making a big deal about how the chimpanzees used for the adverts were carefully kept away from other chimps so that their behaviour didn't get contaminated by real chimps.

My mother - who is no Dian Fossey - was horrified, saying "not allowing an animal to behave naturally, never heard anything so ridiculous in all my life!".  Indeed it was the last time we ever visited.  Thankfully the ads fell out of favour in the 80s as we understood more about ape psychology through work of people like Dian Fossey.

Unfortunately the damage had been done - the PG Tips Chimps never really managed to integrate back with other chimps.  Indeed the current manager of Twycross Zoo is open about the damage that was done.

It's interesting to take time to think about this.  Zoos originally began as a Victorian curiosity where animals from around the world would be collected for the amusement of paying customers - with confined conditions and taunts from the public commonplace.

The PG Tips chimps are a relic of that kind of thinking, where the primary concern is the amusement of humans.  However zoos around the world have been trying to shake that image - now the focus is not so much amusement, but helping with the conservation of endangered animals, creating better habitats and considering animal psychology, and using the animals as ambassadors for why conservation matters.

It's a powerful reminder how to stay relevant, institutions need to be prepared to change, and to question the behavior and practices of the past.

RIP Choppers - and let's hope we've learned our lesson.

Thursday, April 21, 2016

The danger of letting extremes and corner cases dictate our systems

I found myself quite sickened today reading through the news.  Anders Breivik recently won a human rights appeal case against how he is treated in his incarceration.

Breivik is a notorious Norwegian mass murderer who killed 77 people in several terrorist attacks.  Much like Beate Zsch├Ąpe, he identifies as a neo-Nazi - in only the way someone from Norway who was born after the Nazi occupation remotely could.

In my mind, the unrepentant killer is an absolute monster and there is no greater crime against human rights that the commitment of mass murder.  And so the verdict was somewhat sickening to me.

It's been the cause of a great amount of thought all day.  Stories like this often make the news, because we obsess how something in our society functions when we look at the "corner cases", the extremes of society we can't predict.

Breivik himself presents a conundrum - we was 32 when he committed his crime, but in Norway the law system has a maximum sentence of 21 years for any crime (although under the containment clause it can be extended).  They had never really imagined having to deal with a criminal like Breivik before.  Although murder can be found in any corner of the world, it was just still an unprecedented act in the country (ironically) since the Nazi occupation of World War Two.

We seem to be drawn to these kinds of corner cases in any of our systems,

  • The mass murderer and his human rights
  • The silly health and safety rules
  • The social security fraudster

Often the obsession is not to fix the system - but to use these extremes to dictate the norm - which is usually used to support an overeaction that involves penalising the whole system.

We talk much about health and safety rules being silly (and some are interpreted trivially and obstructively), but lose track of the fact that they are in place to save lives in the workplace. And have helped to contribute to a fall in the number of work related fatalities by ensuring a right to safety equipment, and evaluating areas for potential problems.

The fraudster on social security is used as an excuse to make social security more restrictive and punitive.  When the system exists to help people in genuine dire straights (but they're often forgotten in the outraged hunt for the fraudsters).

In many ways, human rights as they stand in Europe are the legacy of Breivik's idol, Adolf Hitler.  After the monstrous acts committed by the Nazis, they are an attempt to legislate against ever seeing such state-fueled atrocities from ever happening again.

They are rights that are supposed to be there to protect each and every one of us.  The hideously bitter pill for me today was the realisation that for human rights to mean anything, they have to apply even to a monster like Breivik.

Today as I brooded on it, I would have loved to have a clause which could mean you could take away the human rights of someone like Breivik.  And then it had a ring of the familiar to it - and I realised that human rights can't be something we're entitled to.  They have to apply to all of us, the best and the worst, or they don't apply at all.

Once you create a clause that allows you to define someone as "not human", you have set a precedent.  You have created a reason to take away a persons human rights, someone else will find their own reason ... and another and another.  Once you have made it law that such rights can be ignored "in some cases", those "some cases" will always be exploited by others.  That's human nature.

Adolf Hitler managed to convince a country that the Jews were "sub-human" - much like I'm desperate do with Breivik.  Then it became the homosexuals, the mentally ill, and before too long anyone who disagreed with him.  We all know how that ended ...

Even in my old homecountry - you just need to look at the Prevention Of Terrorism Act 2005 which used the 7/7 Train Attacks as it's justification.

This act allowed special powers to opt our of human rights act - and would only be used to fight terror, right?  In fact it became one of the most abused laws on the books ...

It's a great example of the slippery slope that once you create the loophole, people will expand it to use it for anything.

In the end though, though Breivik won his case, it was a pyrrhic victory for him.  In supporting his human rights, we totally rejected his ideology - that we avoided taking a step down the path where we can ignore the rights of a human being, even one like him.

Somehow for our society, our systems, our thinking and our lives, we need to be able to cater for extremes and corner cases.  But not let the norm be dictated and overshadowed by them.

It's easy to forget, but when we protect the rights of someone like Breivik, we are making it that much harder to take them away from everybody else.

Thursday, April 14, 2016

Thinking about test strategy ...

As background to this - this year I'm trying to collect a lot of ideas on test strategy.  It's my hope I might have enough material for a workshop on it next year.

Right now some of these ideas are getting structured, but at an early stage - so do contact me below or on Twitter with your ideas and comments.  I'd really like to push further on this!

Last week I was running an evening bootcamp on testing for Summer Of Tech.  It's an awesome opportunity not just to touch base with New Zealand's upcoming tech people, but also to champion a little bit about the fun and mindset of testing.

During the presentation though - I gave my current definition for testing ...

I've been using this slide for about a year - but realised how within that statement is something I'm thinking about more and more recentl.  Something that's been under our noses for a lot of the time.  Maybe we're so busy, blinded and programmed, because we've never really taken the time to explicitly think about it.


Let's look at the standard test script format we've all had to use at one form or another ...

In a test script when we use action vs expectation we're really talking about "something I control" and "something I observe".

It's fair then to sum up a test scenario as using something we control to produce something we observe which can then be compared with something we expect.

All testing is a form of play between these factors - even exploratory testing which we talked about previously here.  We use a heuristic to vary something we control, we use an oracle to set our expectations, and we observe - raising defects if there's a variance.

Exploding the idea of strategy

At it's heart then, a test scenario seems a very simple beast - we control something, something happens ... is it what we expect?

This is probably where testing gets it's reputation for being easy - a single test scenario presented like that looks simple.  Even if we overlook how observation can be a difficult to master.*

The problem is that testing is not the execution of a single scenario.  The skill of testing is coming up with a series of scenarios which cover a system.  Again, people can come up with a scenario or two, but that doesn't make them skilled testers.

To me, being able to come up with a selection of scenarios to cover a product is not easy - and there are no guarantees.  There is such a vast variety of ways a product can go wrong, that it can be overwhelming.  Consequently we often will see members of the testing community talking about how huge testing can become.

Unfortunately that can become problematic.  I'm currently reading Daniel Kahneman's Thinking Fast And Slow, and we've come to the idea of substitution.

Simply put, when we're asked a very complex question, if it leaves us perplexed we will sometimes answer another related question for which we do have an answer.  In his book, when they asked students in one survey "are you happy?" followed by "how many dates have you been on this month?".  "Are you happy" is a complex question.  Put this way around, they found no correlation.

They then asked another group of students the same questions, but the other way around - "how many dates have you been on this month?" followed by "are you happy?".  This way around they found very strong correlation.  The reason being that when the complex question about happiness came up, they substituted a question they had just answered "how good is my dating life?".

Now remember how I mentioned how testers talk about "the possibilities of what you can test" being huge?  That means when a new system is put in front of us and we're asked "what will you test?", that it's a big complex question.

So no surprises, we tend to perform a substitution of our own, and hence we tend to answer the question initially as if it was one of the following questions ...

  • "What do you control?"
  • "How have you tested other systems?"
  • "Where have you found problems before?"

Now none of those answers is a bad first answer.  The problem is when we're falling into the trap of using one of those without realising.  They are each a very good starting point, but if you don't push further, you'll end up doing shallow testing, and potentially missing something important.

"What do you control?"

I'm going to revisit material from a previous article on Back To Basics, where we looked at registration.

Let's take a look at Facebook registration for a change ...

It's easy to take a look at this and come up with a list of the following which are things we can vary and control ...

  • First name field
  • Surname field
  • Mobile number field
  • Re-enter mobile number
  • New password
  • Birthday drop down lists
  • Female / Male radio buttons
  • Create an account
  • Various hyperlinks

When we look at these items our first approach is to create a list of test scenarios which use different data and permutations.  And that's as I've said is a superb start.  Where it's not great is if once you've exhausted ideas for this to consider you've tested everything.

Because you haven't.

What you've just done is a very competent job of functional testing.  You've chosen items that you know are simplest to control, and you've created scenarios.

What you've omitted are items for which you don't know how to control.  I think an important part of test strategy is being able to recognise "I ought to be able to vary X, but I don't know how" - then asking the question "how can I control X?".  Unless there's explicit instructions, we'll often have a blind spot for the "everything else" which defines a product which is often thrown into the "non-functional testing" basket.

What are some examples of tricky things you might want to control?

  • Platform / Browser you test your system on.  Different browsers react differently - installing other browsers is easy.  And you can set up different virtual machines to simulate other platforms.  Maybe you need an Apple Mac (good luck prying it from your architect's cold, dead fingers) - and maybe some devices such as the latest iPhone or tablet?
  • Screen resolution.  When was the last time you adjusted your screen resolution?
  • Server load.  Does the system work well with multiple users on?  You can use a tool like JMeter to simulate a lot of users on your system - simply it's a tool that allows you to control a lot of concurrent calls to your system and measure response time.
  • Hardware down.  If you've got a load balanced system, do you have a scenario with part of your system down?  It's not an obvious thing to control, but you should ask how you can get someone to do that.

Of course in talking about the above I've provided potential solutions - which was naughty of me.  You need in strategy to be brave enough to say "I need to be able to control X", even when you're not sure how.  Ask your team, and around anyway.  Maybe there's a tool out there, or maybe the team needs to build it into the system as part of testability.  But don't be too quick to say "I can't control X, so I won't bother".

At worst, if you can't find a way to control, then put it as an assumption "we weren't able to test around different X".

"How have you tested on other systems?"

Again a super method to get started, we use our experience on a previous system, and copy and paste it over.  This is great apart from,

  1. Your testing will only ever be as good as your last project (what if instead of good testing, you were instead just very lucky?).
  2. If the current system is almost identical to the last one, that's great.  But more than likely there will be differences - and do you need to account for them?

A good example, if you didn't need mobile device testing on your previous project, you're in danger of thinking that can apply to this project - are you sure?

"Where have you found problems before?"

Once again - this is a helpful method, with a hideous blind spot.  If you use only test areas where you imagine there could be problems, what if there are problems in the system you're just not imagining?  How can you think bigger in your testing approach?

If you only go looking for old bugs, the problem is you might not be taking a different path to find new ones!

A useful guide for strategy

Is there a particular approach you tend to favour?  In the early days, I tended to get stuck on "where have you found problems before".  I now tend to obsess a little on "how have you tested on other systems".

When you're looking at a new piece of work, capture all of your initial ideas and try not to think too much about it.  Once this has dried up, think how you've approach this - has it been as if you were answering,
  • "What do you control?"
  • "How have you tested other systems?"
  • "Where have you found problems before?"

Think about the blind spots associated, and dig deeper.  Try answering the other two substitute questions, and see if it forces you to go deeper.  And of course, try out the Oblique Testing cards to shake things up a bit!

Regarding Observation

* If you look at my book How To Test, there is a chapter on observation.  However dispite over 400 readers, no-one has yet Tweeted me to say "where is Chapter 7?", this is one of the deliberate mistakes in the book -

Dan Billing has accused me of overplaying the "smart arse" card on this one ...

Friday, April 1, 2016

Take warning - Microsoft Office Assistant returns ...

I feel the need to take to the internet to warn you about ...

If you've applied a recent update then you'll have notice that the scourge of previous Microsoft Office version, Mr Clippy ...

Yes, the bane of all our lives is back!!!

But help is on hand - there is a way to turn him off if you're on an Edge or IE browser ...

Just look under Settings on your browser ...

Then you'll notice a new button - called Assistant ...

Select the option Deactivate Microsoft Assistant, then hit apply.

They hit the option Confirm, followed by I'm really, really sure.

And there - blessed silence.

Less than impressed at the ordeal Microsoft has turned this into.  I have some friends at the gym who work for Microsoft in New Zealand, and supposedly you'll only have a limited window to turn this feature off, before it's likely to become permanent - as a revenue stream to show you advertising through the assistant on your browser or any related software.

So get in there and turn the feature off before it's too late!

Sunday, March 27, 2016

The other side of comedy ...

There is a scene I remember well from The Birdcage - where Republican politician Gene Hackman is forced to escape the paparazzi, and the only option is to pretend to be a drag queen.  Oh the indignity - at the time I thought it was pretty hilarious ...

It's not alone - Tootsie, Mrs Doubtfire, Old Mother Riley, Nuns On The Run, Some Like It Hot.  Men dressed up as women is big comedy.

Then I made friends with a transexual woman named Violet, and I realised it wasn't funny at all.

How do you know you are male, or are female?  Is it because it was recorded on your birth certificate?  If it because you look between your legs when you're in the shower, and see "the appropriate bit" that reminds you what gender you are?

Although everyone has their different stories, transexuals are people who look at their birth certificate, and the parts between their legs.  And it doesn't add up - they just don't feel that way.  In the time since I got to know Violet, she's not the only transexual friend I've ended up with.  But she certainly opened my eyes to the struggle that goes with trying to follow your true identity.

The common story I hear though is that trying to conform - to follow the appearance and dress code of that birth certificate - feels like living a lie.

Some transexuals are lucky - they put on the clothes and make-up, and no-one is any the wiser.  Sadly not everyone can look so much the role - any they will be the ones who when they walk down the road, people stare, and point, and snigger.

My friend Violet suffered from anxiety - which is not uncommon with transexuals.  The very act of leaving her flat was a terrifying ordeal.  How easily would you cope with having to fend with such unpleasant levels of attention?  It's the other side of the laughter, to understand the tragedy which goes from a desire to just be invisible, so you can hide from the jeers, the prejudice and the hatred.

I have one very good friend now who's going through their own journey in Wellington.  But over the years, I've noticed some others on my commute.  Wellington is quite a tolerant city to live in (I'm thankful to say), and yet even those who are lucky to be "well camouflaged" in their new identity have a tendency to be quiet, and will only whisper in public in case "they give the game away" from their deep voice.

This year there was a male student I'd see on my morning train - and one Monday they turned up to the station dressed for the first time to their chosen female gender.  It made me think of Violet, and I found it incredibly moving - it felt in many ways one of the bravest things I'd seen.

Maybe it's something we all ought to experience once a year - to randomly have to wear the clothes of the other sex, and just not be able to tell anyone the why.  I think though I have some excellent readers - and I like to think you all have open minds and big hearts.  Next time you see someone who you think might be a transexual male or female, just think about all this, and remember the incredible bravery it takes.

Today would have been Violet's 41st birthday - and this blog post is my birthday present to her.

Oblique Testing - it's a resource ...

I've been playing around with the idea of trying to create a set of Brian Eno's Oblique Strategies for testing.  The idea has been around for a while, but I got a useful hook to a fun way of making it work recently.

I've created this ebook about them, although the cards themselves are still "in beta" a bit.  But I'm doing a bootcamp on testing next week, and wanted to include this.  You can print out your own cards from the ebook here.

Friday, March 25, 2016

The Turing report - reporting on scientific process...

Last night's talk from Michael Bolton was as interesting as I knew it would be.  One quote in particular caught my imagination, that I had to put onto Twitter ...

This got me thinking a lot about how reports are given into progress in science.  It's well worth watching such a press release on say "discoveries on Pluto", "string theory" or "artificial intelligence".

The language used is very much like that of a stand-up - and is probably what we need to be emulating in our reports ...

  • This is our journey to date
  • This is what we've learned
  • This is where we're going next

Take for instance the Turing test - this is the test of a machine's ability to mimic thinking by being able to fool someone into thinking they are really talking to a human being (not a machine).

We've made huge leaps in this field since Alan Turing first postulated it.  We're not quite there yet, although we've been "getting there" for about the last 20-30 years.  We probably have a couple of decades to go yet, and even once there, it's a goal that once achieved, artificial intelligence will look for the next goal.

The Turing test caught my attention this morning after reading that Microsoft's recent AI Tay - which was designed to mimic the behaviour of a teen girl, and interact online - has evolved it's behaviour after interacting online.  It now is exhibiting the behaviour of a sex-crazed, foul mouthed, Hitler apologist - so Microsoft had to turn it off (reading this I wondered - did 1st April come early this year?).  I guess you could say her father Bill Gates grounded her after reading what she was saying on social media.

[Perhaps this is also a lesson on the corrupting influence of social media]

In a fit of whimsy, I thought "what if we tried to apply the same rules to scientific reporting as we do to testing?".  With a never-ending focus on "the numbers".

Ladies and gentlemen, I give you "The Turing report" ...

BTW - I like to call this "The Bach Test", after something I learned from James Bach on RST.  If you are trying to get a message across, and someone just isn't listening, and being obstructive, you need to consider walking away (not necessarily telling them to "fuck off" though).  Because when you overly compromise towards someone who is not listening and only has a rudimentary grasp of what you're doing

  • you can never satisfy them because their evaluation of what you're doing is entirely arbitary
  • to make things even more challenging, you're also hamstringing your own efforts.  It's like someone putting a ball and chain on your leg, saying it'll strengthen you up so you can run even faster.  But they never take it off.