Metrics and our love of numbers

I recently went swimming for the first time several years.  As someone who used to live for and love swimming - this seems ridiculous.  I even have an open pool only 1km away!

In my 20s I'd typically swim 2-3 times a week, covering 64 lengths at my local pool in Burton, which would add up to exactly one mile.  However, I felt a bit disheartened with my recent swim.  I by no means expected to be back to my peak ability and speed, but all the same, a target of 40 lengths seemed quite adequate.  However having completed 30 lengths, I just didn't feel like I had another 10 lengths in me, so crawled defeated from the pool.

It was only when I returned a week later that I noticed something ... my local pool in Wainuiomata is 50m in length.  My pool at Meadowside Fitness Centre in Burton-on-Trent was 25m.  I'd actually done the same distance - it also explained why each length seemed to take me twice the time!

On my second visit I didn't make the same mistake.  I made sure to complete 32 lengths of the pool, or 1600m (one mile).  What interested me was that somehow 32 lengths didn't feel quite as much of an accomplishment as when I used to do 64 lengths in Burton.  Even though it's the same distance.

Why is that?  Well it comes down to our blindness when it comes to numbers vs measurement - the subject of today's article.

Money on offer ...

We like to think numbers are a cold, rational, and scientific thing.  But believe me, the way we act on them isn't - when it comes to numbers, we have an irrational attraction where the bigger the number, the better.  For example - if I were to be feeling generous, and said to you dear reader "I'm going to give you some money ... would you prefer 10 New Zealand dollars ... or 1000 Japanese Yen?", what would you go for?

Pretty much everyone will feel the attraction of the Yen figure, simply because it's a much bigger number.  Because I'm asking you, and I've a reputation as a bit of a trickster, you might well be suspicious.  But the larger number is appealing purely emotionally.

In actual fact, the two amounts according to my Google exchange rate calculator are almost equal (currently slightly favouring the Yen).  There's certainly not the gulf of difference between the two currencies on offer that your gut-feeling would have you believe.

Context matters ... but damn the bigger number is a more appealing.

Numbers at work



Okay ... time for a work question.  Your manager asks you how many tests you've executed today ... which would you rather tell them - that you've executed 2 tests?  Or 12? *

Which number will make your manager happier?  Well, the higher one of course!

But you knew that which is why you split the 2 tests you were going to do into 6 parts each.  Again the numbers lose their value if we perceive them as just numbers, and lose their context.

So once again, our emotional desire for large numbers gets in the way of our reasoning.

A world of numbers

We live in a world of numbers - we like to measure and share these numbers.  And sometimes it plays to our vanity a little too much.

For example Runkeeper - anyone know someone who keeps posting to Facebook every run they do, including distance, speed, time?

I've once used a run tracker, but never dared to post the details because I'd taken about 45 minutes to move just a couple of kilometers.  Mind you, it was up this ...

But the data shown on Facebook wouldn't have included the elevation covered.  And even so, again because we want bigger numbers, we're more attracted to running/walking a long distance, than a shorter distance in more challenging circumstances ... because it looks and feels like we've done less.

What's the remedy?

One thing I find quite sad, and sometimes a bit alarming is that a few of the managers and business owners I've worked with who are most "focused on the numbers", are also the people who will sometimes talk with a note of pride about "never really getting maths ... and I dropped science and maths as soon as I could at 16".

I have no problems with managers who aren't good at maths - to me the core part of being a good manager isn't about applying mathematics (but for goodness sake delegate mathematical analysis if you try).  But if you're going to make "interpretation of numbers" the core of your management style, I think it's fair to expect a higher bar of understanding than "dropped out as soon as I could".

Would you want a doctor who flunked then dropped biology?  Would you want a programmer who "read a few books when I was 15 ... but haven't looked since?".

A key part of science in the post-16 curriculum in the UK is "what makes a good graph?".  Which typically revolves around ideas of "units" and "title" ...

• Units is about "what are we measuring".  Always with a question of "is it appropriate?".
• Title is about "what is this a graph of?".  It describes what you're looking at, gives context and helps you understand what you're looking at.
• Error - how accurate is my measurement?  Is it so inaccurate that I can't really take much prediction from what I'm measuring?  [Hint - this is frequently the case in IT, we typically choose things which are easy to measure, but that aren't really useful to measure]

Without these concepts we're tracking numbers, but we're not really sure what we're actually counting - for more information take a look at this previous blog post.

Next time - I'm going to cover the core of what a test report is about, and give some suggestions.




PS: The "number of test scripts" conversation

* By the way, if a manager asks you how many test scripts you've done - it's always a good idea to try and control the conversation, and not reply with numbers.  In the Scrum stand-up for instance (as a guide), you don't go, "yesterday I did 12 test cases ... and today I plan to do also 12".  You say "yesterday I finished off all the login testing, and today I'm trying to get through the account management scenarios".

Sometimes the manager will wave his hands going "I don't want the details ... just the numbers".  But try and have a conversation on what you're actually doing when you can.  It should be important to give a verbal report which is meaningful and reflects your effort (though no lengthy monologues please).