What Gene Roddenberry taught me about "issues that matter" ...

The Making Of Star Trek by Stephen E Whitefield and Gene Roddenberry is an amazing book that I read back in my childhood, detailing the conceptual and physical struggle to bring Star Trek to the screen, from it’s original idea, to the final product.

One piece that stayed with me was creator Gene Roddenberry’s vision of the show.  He felt the public would buy into the fantastic setting as long as they identified with the characters.  To that end, he felt the crew of the Enterprise (with the possible exception of Spock), needed to act and behave just as you’d expect the crew of a contemporary naval ship to behave.

To aid him, he developed a test he asked new writers to take.  When faced with the script below, which of the outlined issues struck the writer as most significant with this piece?

The scene is the bridge of the U.S.S (United States Starship) Enterprise. Captain Kirk is at his command position, his lovely but highly efficient female Yeoman at his side. Suddenly, and without provocation, our Starship is attacked by an alien space vessel. We try to warn the alien vessel off, but it ignores us and begins loosening bolts of photon energy-plasma at us.

The alien vessel’s attack begins to weaken our deflectors. Mr. Spock reports to Captain Kirk that the next enemy bolt will probably break through and destroy the Enterprise. At this moment, we look up to see that final energy-plasma bolt heading for us. There may be only four or five seconds of life left. Kirk puts his arms around his lovely Yeoman, comforting and embracing her as they wait for what seems like certain death.

PLEASE CHECK ONE:
(  ) Inaccurate terminology. The Enterprise is more correctly an international vessel, the United Spaceship Enterprise.
(  ) Scientifically incorrect. Energy-plasma bolts could not be photon in nature.
(  ) Unbelievable. The Captain would not hug a pretty Yeoman on the bridge of his vessel.
(  ) Concept weak. This whole story opening reeks too much of “space pirate” or similar bad science fiction.

In essence this was a test from Gene Roddenberry to see if potential writers shared his core values for his series.

Inaccurate terminology – well, this is a cosmetic issue and easily rectified with a stroke of the pen.  Likewise, with scientifically incorrect, it’s science fiction, so it does stretch science a little.  But again cosmetically can be corrected.

Concept weak … although potentially a more significant point, several of Star Treks best stories started like this – the thrilling Balance of Terror, or the Kobiyashi Maru test of The Wrath Of Khan.

So here, it’s the believability that’s the biggest issue.  If the characters when under duress are behaving in an unrealistic manner, Roddenbury felt you cheapened the drama and the premise.

Lady Gaga - the early years

Yes, Kirk did his share of canoodling with women during the series, even female subordinates.   [Starfleet really needed some regulations on dating].  However when there was an ugly Klingon in his face, he was always a man of action to roll up his sleeves and queue the wrestling music, over “copping a quick feel with the young lady”.

"You Klingon son, you killed my ... no, wait a minute ..."

Fascinating

So where does that fit in with testing?  Well, much like script writers, as testers we really need to understand the “values” of the show we’re trying to help produce.  Star Trek had a series “bible” explaining characters and what the series was trying to achieve, there may be other methods open to us, including just asking Gene Roddenberry directly (or our own version of him, the “visionary person who matters”).

In the scene above, it’s awfully tempting to edit those small issues as you go along.  However, consider this, what if the concept or the characters behaviour is weak?  You are going to tweak the cosmetics, then hand it back asking for a major rewrite.  You’ve essentially wasted your time (and potentially the authors, if you drip fed those changes first), if you’re going to ask for two cosmetic changes followed by a major rewrite.

Even when we prioritise defects, both developers and ourselves only have so much bandwidth to deal with them.  Who really wants to trawl through a hundred cosmetic defects to find the two Major issues buried in amongst them?

It’s the big ones which are the software killers, and what we need to push forward first, and concentrate our efforts on finding.  There’s thus a declaration in my testing mission statement that we will,

"Work to illuminate high impact defects rather than report cosmetic defects en mass"

The team needs to know and work on the big issues as early as they’re known.  When I first get a new build, I won’t stop to defect every cosmetic issue I come across.  I often grab quick notes and screenshots of anything that bugs me as I go along, but if it’s not stopping me, I keep going.  I want to find if the build is delivering the ability to do the key functional stuff.  I can’t do this if I’m stopping to write up every small issue.

An example of this was an issue I encountered a few years ago.  Development was running late – and the work was being done abroad.  On Monday, we received a build, but it wouldn’t allow us to create items at all in our system.  This was, as you can imagine, a level 1 MAJOR issue, that was raised immediately.  Only there was no easy fix it, and it stopped our testing dead in its tracks.

On Wednesday morning, we were closer, but it was still not working.  The foreign development team thought each night they’d cracked it, and each morning we testers found it was still broken.

To break the stalemate, I asked the development team to ring me when they’d done, and I’d test it immediately to give them the instant feedback they needed, so we could start Thursday morning going “yes we know they fixed this”, or they could try again.  I got the phone call at 3am, and went into the test environment.  I could now create items again – the Severity 1 issue was gone.

However I found out that although that issue had gone, there was a new issue, where instead of being able to create up to 12 items for each customer, we were limited to 7.  We’d removed a Severity 1 issue, but now had a Severity 3.

I had to make a quick call.  The issue I was looking at wasn’t big enough to stop us testing later in the day.  The development team had worked into the night in their timezone, and wouldn’t be able to fix this before they went home anyway (not as deploy a build).  If the Severity 1 issue was still there, it’d be different because it would have stopped us from testing altogether.  Besides I was tired, and would need to investigate this more.

I told development they’d done a good job, and I could now create.  There was a smaller issue, but I’d write it up later in the day.

In my mind, this is the same principle in action, prioritising the important defects, getting action on them, but holding back on the less important defects until you’re ready to spend time writing them up and investigating them.

Much as with Gene Roddenberry’s writers, it’s about making sure that you’re spending your time on the core values finding the gaping issues, before adding the polish afterwards.

Thank you Gene for teaching me that …

The Great Bird Of The Galaxy