Tuesday, November 18, 2014

Who "owns" the defects?

Today has been an interesting day at work, where we've explored a couple of questions on "ownership" on our project.

We talk a lot about ownership, but sometimes I get a little tingle in my brain that makes me wonder if we're all using the same definition.  It's very easy for a term to be used so much that different people end up feeling that the word means different things to them, and sometimes I have to admit, I'm the one who's getting it wrong!

It wasn't the first time, and it won't be the last that that I've gone into a dictionary sanity check my definitions.  This is something I encourage everyone to do once in a while!

This was a little tricky as we were talking about ownership in the context of processes initially.  It helped looking through several definitions, and not just choosing the definition you personally prefer, to make sure there's a consensus.  I found the Wikipedia one was the best to talk around,

The process owner is responsible for designing the processes necessary to achieve the objectives of the business plans that are created by the Business Leaders. The process owner is responsible for the creation, update and approval of documents to support the process.

I'm currently overhauling our defect management process on a project trying to keep a balance of (a) reflecting the reality of what we do, and (b) to keep it as simple and straightforward as possible, whilst also giving it a level of flexibility it might need when issues don't fit comfortably.

The question came out "who owns the defect management process?".  I don't just write the process and then go "that's that", and I don't really want to be the "defect management police" either.  The process needs approvals from both the customer, and superiors within my company, but with the definition, "approval" does not equal "ownership".  If the defect management process isn't working, if it fundamentally breaks down or needs nips and tucks, the person who needs to revisit it and work with parties suggesting alternatives isn't the approvers, it's me.  And hence we all agreed that any reference to "the owner of the defect management process" refers to me.

Then came the second question, "who owns the defects?".  In my many years of software testing, I've never heard the question phrased like that, and particularly with the prior discussion defining ownership, I found myself faced with a question which although seemed initially simple, I felt philosophically deserved some exploration.


My first emotional reaction was, who owns the defects - the test manager.  Oops ... the test manager is responsible for the process used to report and even track bugs, fair enough.  But are they responsible for "achieving the objectives" regarding defect resolution?  I've almost put my foot in a common trap, that "testers are responsible for the bugs they find".  Or in other words "you found them ... it's your fault ... why aren't they fixed".  Oh dear.


My second thought sounded a bit too much like agile dogma, who owns the defects - the whole team.  I like this one a lot.  The whole team, from developers to testers are responsible for finding defects, and prioritising work so that the key important defects are fixed before any release to production.  Certainly the whole team in that sense are responsible for working together to achieve the objectives.

Even so there are holes in that idea - in my definition there I talk about "key important defects".  But who decides which ones are key?  As test professionals we can use our experience to suggest which ones we think are more important than others.

But this is bordering on a key mistake many testers can make.  Notice the important use of the word "suggest" there.  We may like to think we make decisions about when a piece of software can go live, and I have read some test exit reports which have said so.


However in truth, all we can do is recommend.  In the end the defects we have represent risks, and we don't wear the risks when we walk away from the project.  This leads me to my third answer, who owns the defects - the customer.  The customer is paying for the software you're delivering, and the defects are part of that system.  Hence it could be said if they are owners of the software, they're also owners of the defects within.  And hence as they own the risk, they should get an important say in the level of the risk they're prepared for - as long as we as testers have done our best to inform them what that risk may look like.  There may be delivery or schedule deadlines which means it's better to release something now and update later if really required (especially if you want to be first to market).  Likewise there might be a desire to "just get everything right first time", even if it means delays.

In the end I've answered the question with three answers.  But it's a good question to explore, particularly with our relationships within the team and customer relationship with respect to defects.  They're commonly associated with us as testers, but they impact the whole team in different respects,

  • As testers we're responsible for finding and reporting defects
  • As a whole team we use our experience to prioritise defect issues from our experience, but should seek customer involvement in this.  As much as possible encouraging them to participate and have a voice in this
  • As a whole team we're responsible for using our time and resources to investigate address and retest the most important of these
  • The customer is ultimately the boss, but we have a responsibility to advise, whilst understanding that ultimately it's not our decision to make

Friday, October 17, 2014

Mental Health 108 - Mentally checking in


It was a week ago now.  My uncle Charlie had died a few days before - it was sad, he'd had a brain tumour which had seemed treatable, but had been too aggressive.  The news was upsetting, but I phoned home and made sure my dad was doing okay with the news, as they were very close.

I thought I was okay.  Because I understand a measure of "human factors", I took a day of leave to be a little reflective, making sure I made buying a condolences card for his family.  I hate buying those cards.  Okay, no-one likes to buy them I know.  But the thing that upsets me the most is you always find them to the right of the "get well soon" cards.  I have before now purchased a condolences card knowing full well we had a "get well soon" card in the post to the family, and feeling dreadful about it.

Me and my wife went to a cafe just over the road from the Post Shop, and the logical part of Mike's mind went "well if you write the card now, you can get it in the post now".  I clicked a pen, opened the card, and everything went white ... I'd been thinking all week about what to write, but now, I actually realised what I was about to write.  And the enormity of it just hit like a brick, and I started to cry in a crowded cafe.  Crying for men is a tough enough subject, but in public!

In truth, it wasn't just Charlie.  The news was sad, he was only in his early 60s.  He was a great character through my childhood who I'd just seen less of in the last decade.  Much like his dad, my uncle Norman, he was a great teller of stories - the best being about how his kids had conned him into getting a dog using the name of his favourite actor, Charlton Heston.  With a name like that, you'd expect a majestic German Shepherd, but what he got was a pocket sized mongrel.  My dad was an only child, so Charlie and his brothers were like siblings to him (although by all accounts my dad was a bit of a terror).

The problem was it's been a tough year in our house - in all there was been five deaths of people quite close to me; two close friends, my mother-in-law, my grandmother and now Charlie.  On top of this, my wife has been diagnosed with acute anxiety which although was a slight relief (originally we thought she was having a heart attack), has been difficult to deal with.

I know I tend to put people first in such circumstances, I know it's a very "man" thing to do (although I don't think it's just men who are like that).  You know someone else will be more affected, and you want to be supportive to them, so you tend to put off your own grief.

And here's the crazy thing - I know about grief.  As a teacher in post-Hillborough Sheffield we had to be aware of grief, and post-traumatic stress - we had special workshops.  I know how we grieve, I know why we grieve.  And when I opened up that card, it was a trigger for everything I was holding back - and despite all my knowledge and logic, it still felt like a ton of bricks had fallen on me.

Knowing doesn't seem to soften the blow one jot.  But here's what it does lend you.  You know it's normal, and even when it happens in public, it's nothing to really be ashamed of.

Even so - it's been a weird week.  I know I've not quite been myself at all - things have happened this week that would normally cause me to roll my eyes and just get on with it.  But instead these events have made me either incredibly angry or really weepy.  Thankfully my wife having gone through her anxiety "gets this" and has shown huge support.

All the same - I know I'm not quite myself, and it's been going on for several days.  Although I'm not feeling depressed and certainly not contemplating suicide, I decided it was still worth going into the doctors today to talk about what was going on.  Never under estimate the power of just going to the doctor to just mentally check in with a professional.  Indeed it was he who diagnosed me with a case of mild anxiety because I've delayed my own grief over other people who were closer than Charlie, but it's Charlies death which has triggered what I'd just been keeping in, and hoping to avoid.

There is some medication if I feel it's very bad (which to be honest, I'm a bit scared of becoming addicted to), and counselling to deal with the grief (which I usually find useful and my preference).  But a lot of it is just understanding it's going to take time - I've been here before with Violet I know, trying to grieve to a timetable.  And yet not knowing how long you're going to feel like this is really scary.

I've decided to blog about this - I'm always terrified people will think less of me for this.  But I think it's also important to try and be open about it.  I'm really lucky to have people like Whirlwind to listen or my friend Lotz who is very open about his experiences with grief.

I talked in my mental health series about Richard's tale and how he knew something was up with him, but only went to the doctor when the depression had taken root.  I've found for myself, a key to keeping good mental health is about mindfulness of ourselves to know that something is wrong, and have the courage and relationship with our doctor that we can go in and say "this may seem silly but ...".  Very rarely is it silly.  If something it bothering you, then it is neither silly or trivial.

Don't be afraid to talk to someone.


Tuesday, October 14, 2014

The doors to my "school of testing" ...


This was an interesting thought experiment today on the train (it's actually based on the lobby to one of the houses at the Aerospace Centre in Farnborough).

First of all, no, I'm not setting up a school of testers ... but if I did ... the lobby would be somewhat unique.

It would have a entrance to the reception with two sets of doors leading in.  The doors to the right would say "use the other side" ... as would the doors to the left.  Randomly each morning the receptionist would unlock only one set of doors.

The doors on the left would have a sign marked "pull", but the doors would only open if you pushed.  Likewise the doors on the right would have a sign marked "push", but the doors would only open if you pulled.

Anyone who got into the reception area would be well on the way to becoming a tester.  Those who didn't get in would possibly be no major loss.  You see the only type of people who'd get in would be those who could read, but would challenge what they read.  Somehow the art of being a tester is trying to push a door that tells you to pull - most other professions within IT would put a sign on the door marked "pull" and go "but why would anyone ever want to push this door?".

Welcome to testing ...



P.S. - If this was a real school, you could post through your applications, but I'm not very sure we'd ever get the mail ... unless we set up a PO Box.

Tuesday, October 7, 2014

Managing Management Relationships - a WeTest experience report

This is a copy of my write up originally printed on the Assurity website here as part of a WeTest workshop that I ran last year.  I'm repeating it here because this is fundamentally how I try and work with people, so nice to put down as part of this blog ...


Like some strange hybrid of James Dean and Marlon Brandon, testing is one of the more misunderstood children of the software lifecycle. Past WeTest workshops have touched upon the friction that can occur between management expectations and testing realities and it seemed worth spending an entire event exploring the relationships between testing and management.

My experience report on this topic occurred in the UK. One of the toughest relationships I had was with a Project Manager of a project already under considerable duress. There was insufficient space to house testers near the rest of the team or even in the same building.

Simply put, testing struggled to play catch-up with the rest of the project. Many aspects of it followed an Agile-esque model, but not being around, we were never able to find out the many verbal decisions that changed the direction of our product. Under such strain, we were seen as the ‘bad news bunnies’ with each problem we found. “We’d be ok if testers stopped finding bugs” was a common refrain, despite our attempts to explain that we hadn’t put them there. It was a de-motivating and fractious time for all involved.

Then I found out my next assignment was with the same Project Manager. Not encouraging. But what happened next was to be a major breakthrough as we were collocated as testers with the rest of the team. This meant that, across the partition from me, sat both the Lead Business Analyst and the Project Manager.

I soon realised that my Project Manager was not quite as difficult as on the previous project and had very real pressures on them from their steering committee.

Nevertheless, they had some very strange ideas about testing. They oversaw some flexible models of software delivery – but they looked at testing to deliver using rigid scripts to check requirements which were unavailable. This was because every script they worked on had involved testers using scripts and graphs. So these scripts and graphs seemed more important than actually ‘executing testing’. Likewise, their steering committee liked graphs and percentages because they were easy to understand in some odd ‘double-think’ kind of way – and because everyone believed that when you are 90 percent ‘done’, you’re halfway there.

To balance this out, I realised that, as much as they had unusual ideas about testing, I had some strange ideas about project management. Just by being in the same environment, I was able to learn a lot from them. I also set myself an objective – to mentor them on the nature of testing as the project progressed.

Somehow we managed to meet in the middle. But the relationship wasn’t an easy one – and at times got very fraught as we seemingly wanted different things.

Whenever we argued, one of my peers would bring in a peace offering the next day, usually something baked, which really annoyed my PM. But it was something I came to respect when they explained, “We’re working towards a delivery. We’re not always going to get on or be best friends. Sometimes you’re going to piss me off and I’m going to piss you off. We just have to work through it”.

It was then I understood that testers and project managers were in a relationship, just like being in a marriage. They were united in some goals, but sometimes we wouldn’t get on and would have differing ideas and goals. Our relationship had to be strong enough to weather such times. It had to be give and take. You try and make the other person happy on the cosmetic things when possible, but hold your ground on the important things. But when you do, you tell them and try to sway them to your thinking. It’s a battle to get yourself understood, but the important thing is to maintain the relationship.

If you stick to your guns and break the relationship by being inflexible, no one wins. The trick is to lure them to your way of thinking. If they have odd ideas, educate them. But listen and make sure it’s not you with the odd ideas. Let them educate you on their problems and try and offer compromises.

Co-location is vital, for all the subtle ways it allows you to be attuned to each other and pick up on each other’s needs. It’s possible to maintain a relationship with your management, just like it’s possible to have a long-distance relationship. But it’s also a lot more challenging and you need to make the effort to have time to catch up – something which came out of the follow-on discussion.

There are no silver bullets, but you need to work on that relationship as you would work on your marriage. The war for harmony with management is much like a battle for hearts and minds. It will be challenging at times, but it’s also worth it in the long run. For me, my ‘best’ and ‘worst’ Project Manager is the same person. The only difference is that we both worked hard to make it work.

Thursday, October 2, 2014

Reviewing More Agile Testing



Working as a reviewer on Lisa Crispin and Janet Gregory’s More Agile Testing was an interesting experience.  I was “recruited” towards the beginning of 2013 when Janet and Lisa began working on a follow up book, for which there was a huge appetite.  When the original book came out in 2009, “agile” (certainly in New Zealand) was a relatively fringe term.  But certainly by 2013 where the bulk of the book was being written it was getting more and more rare to come across people who were unfamiliar with the concepts or had no experience with agile at all.

The world had moved on in the five years since the first book, and teams which had “gone agile” were finding themselves faced with a whole new set of challenges as technology and business approaches moved on.  Indeed, this was seen inside the book itself in the peer reviewers – in the original book Janet and Lisa had emailed each chapter to individual reviewers who were almost ignorant of each other’s existence. 

However for More Agile Testing, they were able to make use of cloud drives to share chapters, and built up a community of reviewers (16 in all) from various walks of life and cultures.  Through this forum as reviewers we were able to have our comment, and even “write acceptance criteria” for what we expected from the finished product.  The challenge for Janet and Lisa was to deliver the goods!

What’s great about Janet and Lisa is they don’t just sit down say “hey, we’re really smart authors, look how successful our last book was, here are our great ideas”.  They wanted a good deal of challenge and feedback on the book.  Although they reserved the executive decision on the book, they wanted it to reflect more than just the experience of just “Janet and Lisa”, one of the reasons they felt the sidebars which tell of the experiences of other testers in applying an idea was such an important part of their original book.  It’s one thing to talk about the theory, but the individual testimony of “we did this, we put this into action, here’s the benefits and challenges we faced” are powerful stuff.  Theory is nice, but as testers we’re always a little “show me this in action”.  And the new book includes contributions and tales from 40 other testers from around the world ... myself included.  That's a hell of a resource of experience there!

I know from myself, reading Agile Testing in 2010, there were elements that originally went over my head at first.  I almost always seemed to take the role of challenging for people like me who can be initially slow on the uptake.  A key thing I kept asking was “do I need to have read the previous book to get much out of this one?” – nope, More Agile Testing can be read by itself, but it’s enhanced if you’ve read the previous book. 

As mentioned above the agile world in 2014 is very different from that in 2009, with many of the concepts which seemed radical to me when I read in 2010 now firmly in the testing “public domain”.  Even so I would occasionally push for “wouldn’t a quick recap of the principles for X help here?”.

Janet and Lisa really thrived from this.  As said before, this wasn’t an exercise in us deferring to their experience, they wanted genuine challenge from their peers to shape the book and make it as relevant as they could.  They didn’t take up every suggestion I made, but often I found myself asking why they’d said something one way, to find there was more to it than I’d imagined.  And there was a huge difference between the initial and final drafts – the fruits of the dedicated reviewing community they’d built around them, and their efforts at authors (did I mention they were holding down day jobs whilst they did this?).

Over the series, we reviewers would sometimes hijack the reviewing forum to ask our own questions to the assembled community.  Some of these will have ended up in the book in one way or another,

  • Was co-location for agile teams a must? 
  •  What do people feel is the etiquette around giving people on an agile team “private time” vs “time it’s okay to interrupt for”? 
  • Experiences with exploratory testing / Kanban / branching testing 
  • Learning through humour – and indeed the importance of humour in the workplace.  I doubt you’ll see that as a chapter, but turned out to be quite important, and will occur occasionally as a theme throughout the book 
  •  The impact of social media on testing.  Again a huge area of change since the first book

For myself it was a great experience – I’m a notoriously slow reader, but could read as fast as the author could put it out.  Being able to ask questions directly of the authors, and occasionally challenge felt almost surreal – we’re used to a book being a “static” thing, not something we can influence.  I’m really pleased with the end result from Janet and Lisa, proud of my own part, and enjoyed being able to share ideas with a much larger group of peer reviewers.

And it could not have come at a better time.  Whilst the book was being written, my major project was transitioning from waterfall to agile, with the help of Nomad 8 consulting.  This meant the book passed the most important acceptance test of all, as it was being written, my team was putting whole chapters into practice.

That, at the end of the day, is the greatest measure of a book's worth …

Back to basics 4: Account self management

You can find the "game rules" for this series here.

Today, we're going to jump in with some oracle ideas about account self management.  Note that some of these ideas as well as "testing expectations" could also form "design ideas" as well ...

Oracles

At my team we have a series of character cards and one of them is as below, "user who needs their account editing" ...


This is a range of functions which allow you to change some of the details of your account.  The bottom line for a company, it really helps if certain very simple functions a user can modify for themselves over ringing a support line for.  It's easier for them, and means you're not wasting money on call centre staff to do basic jobs.

Make it so a customer has to wait 30 minutes on the phone in a queue listening to Cat Stevens because they've forgotten their password, and more than likely the end result is your customer deciding they don't really need your system that badly.


Account Hijacking Danger!

There are some account details which if changed can lead to account hijacking.  This is where if someone uses a machine in at an internet cafe after you and you're still logged in, they could potentially "hijack" your account.  We'll talk a bit about those when we encounter areas where it could be dangerous.


Obviously a lot of the below depend on context of "what service you are delivering" and the level of rigour you need around it.  Lets look at some details you might want to change ...

Changing Name

People change name a lot.  The most obvious one is after marriage when it's tradition the bride change her surname to the groom's.

But there can be other reasons as well - for instance I have to admit not being too in awe of the surname "Talks", as my school life pretty much consisted of every teacher saying at the start of the year, "Michael Talks ... I hope he doesn't".  I thought I had it bad until at University I met a guy called Nicholas Lunt who'd wanted to be a teacher, but decided otherwise "because of what my name rhymes with".

Now if you're Twitter or Facebook or any social media, this should be a fairly easy thing to do.  However I do know some social media make have safeguards - you can't for instance change your name to the name of someone famous like "Kate Middleton", "Arnold Schwarzenegger" or "Donald Trump" without flashing some ID to prove that's really your name.  Facebook also has an issue with the surname "Talks", which it thinks is so rare "it's just a joke".  Meh, thanks.

The difference for this would be if you have an online bank.  For that given the ability for fraud by just changing your name, you'd want some more rigour and a "come in and show proof of name change".

Changing Date of Birth

As far as I know, there is no way you'd ever want to change your date of birth.  Okay maybe you'd want to younger, but there's no legal reason.  If your system offers you to, then this is a bit of a no-no.

Changing Gender

We can get a bit schoolboy giggly about this.  But having experienced the other side of this, and the difficulty of changing gender through friends like Violet, having the world recognise your new gender if different from birth gender is a big deal.  People who do need to be treated with respect and compassion, and not the butt of a joke.

The laws in the UK and NZ recognise the right of people to legally change their gender to that which they associate with, regardless of birth gender, so typically your system needs to as well.

Changing Password (Hijacking Danger)

Yes, it makes sense to change passwords occasionally.  But to avoid the hijack scenario, it's generally good to ask for the old password first to prevent anyone from doing it.  It also makes sense to send an email/text to say the password has changed, just in case you go "wait a minute, I did not change the password on this!".

Changing Email Address / Phone Number (Hijacking Danger)

Your email address and mobile phone number are typically used in "forgotten password" scenarios where you say you've forgotten your password, and a temporary one is emailed to you.  If an unscrupulous party sets the email to one they control, then they're just a set away from hijacking your account.

Hence it makes sense as for password to have this change password protected, but also for there to be a "changed email address" notification sent to the OLD email address.  And similar for the mobile phone number.

Hopefully I don't have to explain why it needs to be the old email address to you - if not, have a good think about it!

Similar logic to this applies to changing your mail address.


Heuristics And Test Ideas

This is the last in a series looking at oracles, heuristics and test ideas.  Our first exercise quite thoroughly set out the details for testing account creation.  The second exercise, regarding logon gave some more room for you to try things out yourself.

This time around, with those examples in your mind, and our expectations above clearly set out, you should be ready to fly this one solo!

Do use the comments below, or find me on Twitter to say how it went.  Hopefully you've found this useful.  We're really good at talking about testing, but sometimes it's useful to have a few fleshed out examples, especially if you're relatively new to testing.  Typically as the work we do is confidential, it's not like we can even "just take what we do home and post online".

Tuesday, September 23, 2014

Back to basics 3a: Mindmapping that last set of tests


You can find the "game rules" for this series here.

Yesterday I went through a lot of the options for creating a series of test ideas for our sample login page.

Sadly for an article like that, a lot of things had to, by necessity, be written out the long way.  As a rule of thumb looking at all those test ideas, I estimated there would be about 3 days to script it all up formally (depends on the level of detail you add).

A lot of people are proposing using mind maps to guide testing, but within Wellington, most people have come to hear of their use through talks by Aaron Hodder which you can read about here or here.


Yesterdays piece took about 90 minutes all told to write.  I managed to put much of the test ideas into this mind map in about 10-15 minutes.  Much quicker.  Although of course the mind map lacks detail, and the "discussion of why we're testing" commentary I needed to add to yesterdays document (because it's written to be an instructive guide).

Turning a test idea into a limb of a mind map is definitely "a skill" over a set of rules.  Some of those test ideas were a couple of sentences long, and a limb of a mind map won't handle that kind of detail.  You've got to be able to condense it down, put something that captures the essence of the test idea.  But will make sense when you look at it again in 2 days going "what did I mean there?".

If you don't have a Twitter account, do get one.  Twitter with it's 140 character limit is great for getting you into the habbit of condensing complex ideas into a sentence!

Aaron suggests using a mind map to help plan out and guide your testing.  If you print it out and tick/scrub out elements as you test them.  That way it shows what you've done, and what's left to test.  A useful trick, and one worth bringing to your attention.

Also, show it around (maybe even put it on the wall), and ask people if they have any better ideas.  After all, two heads are better than one - unless you're this guy ...