Shakespeare's lost (test) scripts ...

Shakespeare - great scripter of plays, not so great at scripting tests it would seem.

This started as a bit of fun after watching As You Like It the other week, and through reading William Shakespeare's Star Wars (basically Star Wars as Shakespeare would have written it - fun book).  What you Shakespeare's test scripts look like?

Test Script The First

Action

On approaching the steely sentinel, who lies in wait, ever watchful as Heimdall in ancient times stood in silent vigil over Asguard, render unto him both thine identity and a word most secret.  A word, which hath been arranged in private and ascertained that even in dire peril not to be diverged to man or beast that may roam this land.

Response

Avast, avast!  Look forth as the heavens rend themselves, as a choir most angelic in voice chime "hallelujah" and bid you welcome indeed with great merriment and triumph within.

Or in other words ...

You provide correct details ... you're logged in.

Test Script The Second

Action

Before you lurks sternly a being who in nature unto thee is as hostile as the Minotaur of those ancient tales unto Theseus,  and unto thee doth a grim challenge issue.  Unto him thou shouldst render your identity, and a word of secrecy most counterfeit.

Response

The Minotaur in most dire countenance doth bellow and rage, and in words most stern warns that thou shalt not be permitted to pass.

Or in other words ...

You log in with false details.  You're not let in.

Test Script The Third

Action

As intimidating as a three headed Cerberus stands the challenge betweenst thou and thy goal, to assail to the fortress beyond.  Unto him thou givest thy name and a password contrived to be most treacherously false.  Onward thy web of lies are repeated - once, twice, thrice.

Response

Upon the third falsehood, barks the dog most foul.  For ahead dost a portcullis descend from the loft of heights to bar thy way, each ratchetty clank being the sound of thy doom and abandonment.  

Oh woe lament, as a voice from the darkest pit deep down, a voice wreathed in brimstones own stench doth call out, "thy art locked out for thy falsehoods in great number.  Lament, oh lament ... and contact the help desk".

Or in other words ...

You attempt too many times to log in with false details.  You're account is locked.

As always, behind my piece of fun, there's a bit of seriousness.  During the last week at work I've tried to hammer out a bit of a strategy document around "scripts or exploratory testing?", and if you are going to script, how much detail is really required?

As a preference I do of course prefer exploratory testing, especially for smaller one-person projects, and I outlined my team's approach a few months ago in the linked piece.  Even so I do like to work out what approach is going to be best for the project underway - and there are many, many factors at play for this.  Even in that article, we discussed that there were a couple of areas where we did find scripts quite useful - typically in a barely touched, but quite technical couple of scenarios which took a bit of set-up and were far from obvious.

When deciding an approach for testing, I find it best to consider doing test scripts first.  Not because "this is the best approach", but partly because this is the usual expectation with many non-testers, coupled with the fact that there are many constraints where a scripting approach is just not going to work, and if you proceed then your testing is going to be highly inefficient due to these factors.  It's best to have these discussions up front to make it clear why you feel scripting is unsuitable for your project (I don't think it'll always be like this, I hope in 5 years time exploratory testing will be so much the norm, you'll get odd looks if you say you need to do scripting).

As discussed during the previous exploratory testing article, the most obvious limitations for test scripts are,

• They need time upfront to script
• At the point you start scripting, the requirements need to be witten down, and hopefully fairly unchanging.  If they change a lot - guess what you need to do with those scripts?  You can get around this by mapping requirements to scripts, so if something changes, you can map out what needs to change.  But all this comes at a huge cost - it takes a lot of time and effort for this, and you have to keep it up to date, or else you're back to square one.
• Once you actually get your hands on a system, you immediately come up with a few new ideas for how to use the system that never occurred to you from reading the requirements alone.  I wasn't too keen on the concept of tablets - the joke was I was so OCD about my laptop screen, that having a screen you touch on purpose was a no-no.  I'm no dunce, I saw friends ones, and I knew what they did, and how they worked.  However when I got my first Kindle Fire, I did a lot of playing around and trying things I just couldn't have imagined even with all the reviews and how-tos that I'd read.  Actually having the device hands-on opened my mind to ways to use it I just couldn't imagine from a pure theoretical level (and that's all scripts are).

If you are using pre-scripted ideas, getting an idea of the right level of detail needed for a script is important.  The more detailed the instructions, the more there is to write a new script, the more effort you have to undertake to create a new script.  And as Shakespeare's test scripts show, more doesn't mean better.

If you really, really have to use them, brevity is the key.  Especially if you're using a customer website, simplicity is vital - remember your customers have to use your systems without any script to guide them!.  If it's not obvious to you, it won't be obvious to them.  If you have specific (and little used) back end  admin systems to test, you might find you need more in-depth.  Using the right level of detail is important.

But even with those admin areas, as discussed in my other piece, a handbook of some kind covering off what users need to know probably has more value than detailed scripts.  With a handbook, you can give someone the book, put them on the system, and encourage them to play around and get used to the system.  A good handbook should include a commentary to teach the user the values of the system - that doesn't really come across in many scripts.

Of course if your tests really can be summed up simply by "you can log in", "incorrect details means you can't login", "three incorrect details locks account", it obviously doesn't make sense to have dedicated test scripts with one line scripts (oh you laugh, but I've seen just that in many a test domain).  Instead having some form of matrix to show when you're run/not run tests would be better, with a notes section in case you have any significant problems you need to record.  Which then makes you wonder, where do test matrices stand?  They have some form of script properties, but also some form of exploratory properties.  Does it really matter if they're more script or exploratory testing, as long as they fit the needs of your testing?

At the end of the day, there's no right or wrong - even in my piece on exploratory testing, I talked about areas where we found it useful to keep scripts, because the testing of system errors wasn't particularly intuitive.

There's a core insight that separates good testers from bad ones.  An inexperienced tester will go "we used X on Project A and it worked well, we should use X on Project B as well", they just seek to copy and paste their last strategy to their current project, no matter how different A and B is.  An experienced tester will go "we used X on Project A, but because of these differences, I'm not sure if it'll work on Project B.  We need to consider another approach".

What we need to strive for as testers is an understanding of the factors, strengths and weaknesses of detailed scripts, high-level scripts, test matrices and exploratory testing.  And use this understanding to determine the needs and drivers for our current project to find the best solution for the constraints.  Yes - even exploratory testing is not a "best practice" for all testing - just the best fit for many kinds.

An open mind sees more ...