Wednesday, March 23, 2016

Software testing - when we're preaching to the choir ...

I have very mixed feelings about a local testing event coming up tomorrow.  I should really be excited, it's yet another opportunity to listen to a world renowned software tester - this time, Michael Bolton, whose work and humour I really enjoy.

So why so down in the dumps?  Because of the topic ... test reporting and metrics.  Because I realise "we have to have this conversation AGAIN".

I've posted about test metrics recently here.  And here and here and here and here.  In the 6 years I've been on Twitter, it's been a constant topic in the testing community.  But it feels like we're not done with it yet.

I'm of course excited that lots of testers in Wellington will get to see Michael speak, and some who will never have thought about it, will have their minds widened a little more.  I've looked at the notes for what Michael's likely to talk about, and excited that he'll talk a little about realistic reporting you can do, based heavily on the work he's done in collaboration with James Bach.  You can find the slides to Michael's talk here, and a talk from James on reporting here.

However I was taken back to a comment from Fiona Charles before she spoke at Let's Test Oz in 2014.  I'd just shared a lift with her, and she was about to give her talk "The Battle For Hearts And Minds".  So of course I "wished her luck" (like she needed it), to which she said "thanks ... but I get the feeling I'm very much preaching to the choir".

That off-the-cuff remark really stuck with me, and to be honest has worried me ever since.  I write this blog, and a couple of books.  I present about testing.  I have been called at work a champion for testing before now.

I do get some feedback from other testers especially saying they love my work, and how I explain things.  But I'm constantly worried I'm not breaking new ground the people who don't automatically love what I say or the message of Context Driven Testers and their allies.  It's easy to get acclaim when you're preaching to the choir, but you're not moving forward.

We do need to break the grip of metrics in our craft, which is getting like saying to the guy operating an MRI scanner "I thought you'd use leeches, my old doctors normally use leeches".  That means testers as a community having meaningful conversations about test reporting.  And that means you!  To quote The Lorax,

For me my action plan is as follows,
  • Plan testing around measures of reporting which do not use metrics, but build reporting into the structure of the plan from the off.  I've promised an article on this a month ago, it's not yet in the state I want.  I'm hoping Michael Boltons talk will help me out a bit.  But it has to be said, I'm not always able to win this argument and have the freedom to implement (so don't lose heart).
  • Keep having meaningful conversations about reporting with my testing peers and managers.  Most importantly showing them ways they can see progress and problem areas.  Even when we have graphs forced, it's never the graphs which show the issues.  Education is important.
  • I invited several people I have worked with / are working with to Michael Boltons talk.  Non-testrers who I felt would benefit from it.
  • Engel Consulting is bringing Michael Bolton back to NZ in October to do RST For Managers.  I of course would love to go.  But perhaps more important (I'm aware I'm in the choir), I need to try and talk some of the other (non-testing) managers I work with to go because they need to understand how the Rapid Software Testing approach we've embraced works best.
That of course is great, but more than just me, or Michael Bolton, or James Bach, or Fiona Charles.  It needs all of us talking about the subject in a meaningful way.  Michael, James and Fiona can't be everywhere, and they can't win every battle  Testing is a profession we love, and we all need to champion the things we love, wherever we can.

Be part of change!


  1. Yeah, you do seem to be at a very familiar junction. I have been saying for years, that all our efforts to change testing for the better cannot be accomplished if we do not move beyond the realm of testing and testers. And yes it often feels like preaching to the converted. We need to influence the whole SDLC and get other people to see the fallacies and dangers of metrics. Why the wish for metrics actually prove a lack of leadership, trust and knowledge (i.e. are a good indicator for other issues the project has).

    BUT in order to do so I know that I need to be firm in my grasp of the logic. In order to do that I do need to re-visit the topic again and again. That might just be me but it often happens that I do get new insights. The whole metrics thing has been covered well and it is pretty much "solved" but that doesn't mean the implementation is done. So that is the phase we're in. If you look at I think we're at the end of the Early Adopters phase. We now need to take it to the default level.

    So yes, other people but us need to attend the testing courses. That would actually get some traction. But honestly, do you believe they would ever do that? One thing I rescently read is how testing actually is not an IT discipline and that struck a chord. If we move the perception away from us being "weird developer/BA derivatives" but that we are philosophers, investigators,... (still looking for the right description here), then we actually might make some headway in the minds of others. Maybe then finally that call to measure something that is exploratory, artistic and artsy will become less.

    1. Absolutely +1 to what you're saying.

      And interesting to read your point. With my son doing history & critical thinking, and being driven (as we talked previously) to hunt and provide evidence for any statement he makes - I think he's being better prepared for a future career in testing (if he chooses), than from any current IT degree that I know of.

    2. Buat saudara punya permasalahan ekonomi: hub aki santoro,karna saya sudah membuktikan bantuan aki santoro yang berminat,hub aki santoro di nomor 0823 1294 9955, atau KLIK DI SINIa

    3. Buat saudara punya permasalahan ekonomi: hub aki santoro,karna saya sudah membuktikan bantuan aki santoro yang berminat,hub aki santoro di nomor 0823 1294 9955, atau KLIK DI SINI

  2. There's another option - look at the tech meetups near you and offer to do a talk on testing (or some aspect of testing you think may be particularly relevant).

    I've been pleasantly surprised by how interested devs are in testing talks - I did a talk aimed at devs working in companies without testers in a tech meetup a few years ago (which turned into more of a discussion session) - I thought I'd done poorly in that we jumped around all over the place and never went very deep on anything, but the organiser later told me it was a really popular session.

    Who knows, once you've whetted their appetite you may see more coming to the test meetups or courses?

  3. I second Anna's experience. Last week I gave an introduction to exploratory testing workshop at a local testing meetup which is open to anyone. We had two developers there as well who didn't attend before so this topic is clearly of interest to non-testers as well.
    The one person I couldn't get to go for our hands-on testing part was the 30 year experience tester. The devs had good approaches and were keen to learn.
    I also had the feeling that I didn't go as deep as I'd have liked but we can schedule additional sessions , for example for note taking using mindmaps, more heuristics, managing the test effort using SBTM, etc.
    So, come on, you know you want to, organise a 2hr workshop locally and invite not just testers but other roles as well. :-)

  4. I remember thinking the "preaching to the choir" comment when I was at TestBash this year and that it'll probably be along the same lines at Let's Test this year. So I'm hoping to get in front of more "traditionally-minded" testers (waiting to hear back regarding a abstract).

    Also, assuming you don't already work with people who don't know about CDT, how do you get them to come to meetups etc?

    1. If you have any switched on recruiters locally - inviting them and asking them to publicise to their mailing list will often bring in people who have no idea there's this whole community out there. Be kind to them (i.e. discourage "oooh, EVERYBODY knows *that*" snootiness, and they may come back.)

    2. I've never actually thought about inviting recruiters to testing meet-ups, but that definitely makes a lot of sense :)

  5. This comment has been removed by the author.

  6. Ass Wr Wb Saya ingin berbagi cerita kepada anda bahwa sy seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU LUSI yg dari singapura tentang AKI yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey 4D dr hasil ritual/ghaib melalui jarak jauh dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.270.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI saya juga sudah bisa se sukses ini, pesan AKI yg slalu sy ingat setiap manusia bisa menjadi kaya, hanya saja terkadang mereka tidak tahu atau salah jalan, Banyak orang menganggap bahwa miskin dan kaya merupakan bagian dari takdir, Takdir macam apa? Tuhan tidak akan memberikan takdir yang buruk terhadap kita semua, cobaan yang Tuhan berikan merupakan pembuktian seberapa kuat Anda bertahan di dalamnya. Tuhan tidak akan merubah nasib Anda jika Anda tidak berusaha untuk merubahnya sendiri, Jadi teman2 yg dalam ke susahan jgn pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha. AKI MAULANA adalah guru spiritual terkenal di indonesia yg bisa melakukan ritual ghaib seperti:
    1.Pesugihan bank ghaib
    2.Ritual tembus togel/lottrey
    3.Transfer janin
    4.Pelaris usaha, jodoh DLL
    jika anda ingin mengubah nasib seperti saya silahkan ☎KLIK DISINI PESUGIHAN DUNIA GHAIB


    I want to share a story to you that sy a migrant worker from malaysia and i accidentally open internet and i see the comment LUSI mom from singapore about AKI who has helped him become successful and finally i also try to contact him and alhamdulillah he would help me to Give the number Togel / lottrey 4D dr ritual results / ghaib through long distance and alhamdulillah it really proved translucent and won RM.270.000 Ringgit, now I am back indon buying house and train even though sy just housemaid in selangor malaysia, sy very thankful To AKI MAULANA and do not forget to give thanks to ALLAH because through AKI I also can be this successful, AKI message which slalu sy remember every human being can become rich, only sometimes they do not know or wrong way, Many people assume that poor and rich Is a part of destiny, what kind of fate? God will not give a bad destiny to us all, God's temptation is proving how strongly you endure it.God will not change your fate if you do not try to change it yourself, So teman2 deep into tusah jgn never despair, when it's time god must be the road to where you want to try.AKI MAULANA is a famous spiritual teacher in Indonesia who can perform the occult rituals such as:
    1.Presugihan bank ghaib
    2.Ritual translucent togel / lottrey
    3.Transfer fetus
    4.Paris business, matching DLL
     If you want to change my destiny please ☎CLICK HERE WORLD WIDE GHAIB