Sunday, July 7, 2013

The ballard of LEEROY JENKINS, software tester ...

The last two days at KWST3 have been another interesting experience.  There were many worthwhile, worthy and educational experiences shared, and once more I feel about a dozen articles of inspiration are ready to spin off.


A huge, and fun take home from the event actually came from Oliver Erlewein on Twitter, who together with Aaron Hodder, introduced me to the tale of Leeroy Jenkins,


The youtube video is about a group called "friends for life" who play the multiple online roleplay game World of Warcraft.  They are outside a battle level which has given them trouble in the past, and wondering if to just go around it.  However it contains something that will benefit one of their group (Leeroy Jenkins).


So, they have a cunning plan.  Their leader gives instructions that he'll do an devastating area attack, which needs to be followed up on, with a similar one once done.  One after another to "shock and awe".  Meanwhile they will just try and get in there and out. Their wizzards will provide support by doing magic behind them to make their attacks more effective.   [You'll have to forgive me as I don't do World of Warcraft, so my terms there might be a bit off ...]

Their leader thinks it's a good plan, even though it only is calculated to have a 32.33% (recurring) chance of survival.  They are working on this, when one of the party (Leeroy, who is ironically the only one who will benefit from this raid) gets too impatient, and just goes "okay let's do this ... LEEROY JENKINS" screaming a battle cry and rushing in.

Caught off guard, the whole group run in after him, repeatedly chanting "stick to the plan, stick to the plan", even though the plan called for that devastating initial attack before everyone followed through.  There is a scrambled attempt after chanting this for a while to attempt to communicate, which quickly alters to "goddamn it Leeroy, why do you always do this" for the rest of the game, as the party are picked off one by one.

I have found it a really fascinating insight into the whole idea of "creating a strategy and following it through".

Of course on the internet, Leeroy Jenkins is seen as a bit of a fool, you don't want to be in a team with, because he's quite selfish and impatient (his group go "why do you ALWAYS do this Leeroy").  And there's a huge amount of truth to this.

However, if you haven't watch that video yet, I urge you to watch it now ...

The Planning

The leader says that the room has given them a lot of trouble in the past, and he comes up with a plan.  Although everyone has done the room before, he doesn't ask anyone else if they've got any other ideas.  There's no "retrospective" of "should we try this or should we try that".  Consequentially he never asks anyone for any input on anything that could improve it.  It's his plan, and he thinks it's a good plan, that's never put out to the rest of the group, even with the 32.33% odds of survival.

The problem is, a lot of "plans" and "strategies" seem ideal when removed to a few points, and not open to input from a larger group of the realities and obstacles ahead.

The Battle of the Somme, was to be a breakthrough battle during World War One for the British Empire.  Instead it led to the worst day in British military history where 20,000 soldiers were slain in a single day.

I'm sure if there had been PowerPoint back then, it would have looked something like this,


Stick To The Plan, Stick To The Plan

History, like with the Somme, is filled with battles which "seemed a good plan at the time", but rapidly became unstuck.  Leeroy Jenkins team follow in after him chanting "stick to the plan", even though the first part of the plan has already unravelled.

It's easy here to blame Leeroy, but remember, the plan only had a 32.33% chance of success.  So in all likelihood, was going to end in the same kind of result, and everyone in the party was comfortable with that.

Leeroy acts as a catalyst starting a bad situation.  However, the team tries to communicate to each other when they get into difficulties, and are all talking over each other.  They soon degenerate into blaming and cursing as they're picked off.  It's actually this failure to communicate (over Leeroy's impetuousness) that guarantees their failure.

Something about that video that strikes me from my time at KWST3.  Sometimes we can learn the worst possible lessons from things going bad.  The lesson that team, and especially the leader was jumping to was "goddamn you Leeroy" or in other words "you ruined my beautiful plan".  There is an over reliance on that plan, and not enough on the team.  Again, let me say it again, a plan which was seen to ONLY have a 32.33% chance of success.  Those aren't great odds (and of course I dispute them and even why they were needed).

One of the reasons the Somme went so wrong was the insistence of following a plan.  As above it was pretty simple, kill the enemy using an artillery barrage, then occupy their position.  It soon became clear they'd failed in step one, as the artillery had failed to kill enough Germans (who had deep, underground concrete bunkers).  The plan pretty much depended on this.  But the captains in charge stuck with it anyway ... and this is how disaster happened.

For myself, I have often written many beautiful test plans, which once we've got started I've realised reality has got in the way of.  As someone who believes in the Agile manifesto though, I do believe in,

Responding to change over following a plan

However like the leader of "friends for life", I've known many test managers who have cursed reality for failing to follow their beautiful plans.

Leeroy Jenkins - some lessons for testers

Act like there is a Leeroy Jenkins in your team.  Try not to overplan, but make sure you are including everyone in the discussion, make the plan a team plan, not your plan.

If the team does feel consistently let down by a Leeroy Jenkins, maybe it's time he found another team.  But make sure he's not being used as an easy scapegoat.  Maybe he has a point about trying to not overanalyse, and get stuck into the job at hand.

Most importantly, make sure your team knows how to talk to each other.  Before, but especially when you're in the lair of the enemy.  If someone needs help, let them speak, and respond to it.  Keep responding to the situation and ask for updates, but don't let it become a garbled chatter of panic. Ask people for updates, and try to triage requests.

Chanting "keep to the plan" won't help, and cursing Leeroy Jenkins will just make the air toxic with defeat.

If things don't work out, don't hold up your plan and being perfect, and your team as being the reason for failure "because they didn't follow the plan enough".  Things change "how dare you get eaten by giant spiders when you were supposed to be casting divine intervention ... that wasn't in my plan".

Friday, July 5, 2013

KWST3 - Learning about software testing ...

Today is the first of two days of the Kiwi Workshop on Software Testing peer conference #KWST3, which this year covers the topic of,

“Lighting the way; Educating others and ourselves about software testing -  (raising a new generation of thinking creative testers)”

This topic is of huge interest to me - I'm not only a qualified and certified teacher of science, but also have spent a good amount of time in higher education, even being self-taught in software development (and before you ask, I was a damn fine programmer as well).

Helping to develop in others an understanding of what testers do and our methods is I think a core part of the profession, whether it's developing junior staff, or demystifying our role to project managers and developers.

So what is a peer conference?

In other types of conference you tend to be lectured to by someone who has found an idea or strategy.  They use lots of powerpoint slides, and with luck a few questions are possible afterwards.

A peer conference, however, is much more interactive.

It's based around the idea of experience reports.  Any attendee can choose to give a 5-10 minute talk about an experience they've had which relates to the main topic.  At the end, that experience is given over to the group, who can explore that situation with a series of questions.

Unlike the standard conference where you're just really pooling the speakers experience, this means that the situation is expanded and explored by the sum experience and intelligence of those in the room.  Thus a peer conference is much more exploratory of the subject matter.  You can't look at an agenda and know where you'll end up by lunch, the end of the day, the end of the conference.  That's both exciting and scary ...

Numbers at a peer conference are always tricky - you need enough people to have diversity of opinion (otherwise you'll get nothing but lots of agreement out of the sessions), but not so large that people feel they don't have a voice.

The Opening Experience Report ...


This year, the opening experience was given by Anne-Marie Charrett who is a software testing coach based in Australia.  She has been teaching testers for years, and had experience at the University of Technology Sydney (UTS).

When approached to give a formulaic course at UTS, with her exploratory testing experience, she felt was too much a repeat of standards.  She found she wanted to rewrite it, and sought permission.

Before long, she was making videos, and trying to make sure the content was appropriate, and something she could stand by.  Used AST materials heavily as an influence.  But wanted to make as much hands-on as possible.  Wanted to make it experiment based, as testing is about "doing stuff".

She did the PSL course with Jerry Weinberg - which is about giving a space for learning.  About individual development over "following a course scheme".  Wanted to create something similar but around testing.

An important part was to create a space to work in - where they could be immersed in the exercise, and encouraging people to ask questions. Encouraging students that it's not about giving answers, but finding the right questions.

Some topics she covered over using some exercises and some homework,

  • Mentally modelling software - how do you understand about a product, going far beyond just requirements (if they are even available).
  • Critical thinking - how do you challenge a product?
  • State models
  • Tools and testability
  • Exploratory vs scripted testing
  • Reporting and communication

Challenges

  • The students themselves were used to just sitting in the classroom and given information and lectures.  Being student-centric, it was based on them.  Students needed to learn and develop their communication.
  • Students were postgraduate IT folk, who were more interested initially in development than testing.
  • Dealing with Unversity bureaucracy - wanted a scheduled course they could measure and examine.


She got them to go "into the wild" and into Sun Corp to test and raise problems on live projects. Sun Corp helped to mentor and develop, and students loved and got a lot out of it.  Many had changed their mind on whether they wanted to do testing as a career.

This feedback from students and companies has shown she's onto something, and the course is providing a real slice of valuable skills for a career in testing.