The lion value system ...

When I was working reviewing More Agile Testing for Janet Gregory and Lisa Crispin, there was a whole forum of discussion which was running parallel to the book creation.

One topic we explored though was getting the balance right.  Everyone knows one of the great things about agile teams is that they interact and they talk about issues as they come up.  They're highly social.

However at the same time, there are stretches where people need to focus, and be uninterrupted for a time.  How can you get the balance right?

Back at Kiwibank we had an interesting system - our team had worked hard to think about what our "tribal identity" was.  In the end we thought of a lion as a mascot - because they represented the courage, ownership and attitude which we felt was core to what we wanted to achieve as a team.

Each of us had a little plush lion on our desk.  When you needed to be left alone, he was placed strategically to tell the rest of the team (although new people would have to be told this rule quite quickly as they'd be forever asking "what's with the cute lion" to the person who expected to be left alone).

People weren't allowed to have this lion on display for longer than half a day without explaining - it was there to be used as needs be.

Likewise if we had a meeting, and there was something you needed someone to know was really important about, and you might need them to listen to (especially if you weren't the loudest voice in the room), you'd bring the lion with you to the meeting.

In a way, the lion represented a kind of social contract between team members, and it's something I'm looking back at and wondering if it's worth (and fun) implementing on my new team ...

Here Be Dragons - Critical Thinking film

We've been talking about critical thinking a lot in part posts.  I am actually hoping this year to explore aspects of critical thinking in more detail, especially looking at how critical thinking impacts testing.

I was introduced to the term "critical thinking" really in the RST course run by James Bach.  He has a lot of areas he's explored critical thinking for testing, and I'd like to build on that - or at least find my own way of describing the process.

Until then, I want to point you in the direction of this video "Here Be Dragons" which explores in a short 40 minute form some of the fundamental concepts of critical thinking.

Enjoy ...

Confirmation bias in testing ...

In my previous post I explored about some mistakes I made looking at the New Horizons mission to Pluto, introducing the concept of confirmation bias.

Like many people, although I was vaguely aware of the concept, it took James Bach's RST course to formally introduce me to the concept.  As I discussed in the previous article, our eyes will sometimes see what our mind and emotions want us to see.

In fact it could be said that the very profession of testing occurs because of this effect.

Take a developer who is writing a registration page for a new system.  He goes through it, provides his name, email, date of birth, and it taken to a "Welcome Stuart Cook!" message.  He concludes "it works for me" and moves onto his next exciting coding task.

Can we really say at this point that registration has been "tested"?  The developer enters some data, he sees what he want's to see - he gets a "welcome" message - and to him this confirms that everything is working.

We have all occasionally fallen into this trap (I know I have) - one test does not cover everything.  Being aware that we have a tendency through confirmation to jump to "that works" is a key part of the RST course.

Entering some details, and getting "Welcome Stuart" is a good start in testing.  But we're not there yet.

Is it for instance just a message?  How do we know for sure that account has been created?  We could check the database, and maybe do a login, to confirm we can use the account.  If Stuart hasn't made the login page yet, then maybe we will have to settle for checking the database.  You were going to check the database weren't you?

A minor thing - but what if we create an account for someone who isn't called Stuart - would we still get "Welcome Stuart" as a message?

Then there's the case of "what if we provide really long, diverse fields for answers to the registration questions - what happens then?", plus of course "what if I provide junk answer to the registration page - does it still make my account?".

All these many heuristic ways of testing a registration page were covered back in my Back To Basics series, which is worth a recap.

Sometimes as testers we're seen as being a bit tricky and difficult to work with.

When Stuart the developer checks his name in the scenario, he says "it works" and moves on.

But when we as testers do the same test we say (or rather we should say) "I have some confidence that when I provide details similar to the situation described that the system should work as expected for the scenario provided".

Isn't that written in a lot of safety language?  Some in software development feel uncomfortable.  Lets take some time to explore the phrase I used, and why I used it ...

"when I provide details similar to the situation described"

I provided the following details - they include first/middle/last name of a certain length using certain characters, a certain date of birth, and a certain password complexity.  If the system is presented with data which includes,

• Characters I didn't include
• Strings which are much longer/shorter
• Date of birth which is significantly different to that I used
• Password which radically differs in structure

Then I will be less confident of the outcome of the system, as these inputs differ considerably from the scenario that I used.

Are you comfortable with that?

"the system should work for the scenario provided"

If I've tested the "account creation" of a registered user, I cannot then say I've also tested the "account rejected" scenario.  This is pretty basic, but again in our confirmation bias, if we see an account has been created we can jump to "well this works", where there are really some scenarios that you really DO NOT want it to work.

"I have some confidence"

What a weird statement to start with?  Surely you're sure aren't you?  Okay - I take the above points, but you've created a Stuart Cook user - surely you're 100% confident that if provided with the same information, that you can create a Stuart Cook account?

Actually no - indeed if you've already provided details for a Stuart Cook account, then surely providing the same identical details will mean an error - you can't create two identical accounts after all.  And if you vary the inputs, then there is a level (however minor) of uncertainty.

Okay - so let's say we delete the Stuart Cook account after you've made it.  Surely you're 100% confident you can make it again?

Nope.  One thing I've learned recently since working on highly available test rigs with load balancers.  Sometimes you can get the most frustrating kinds of errors.  You try something and it fails - so you get a developer over, and run it 4 times, each time it passes.  Your developer walks away in disgust, and the problem comes back.

Rather than being transferred to the tester funny farm, it turns out that the problem you're experiencing happens only on one server, and just by pure luck, you've been pointing to the wrong server every time you've been on your own testing!

So what's the cure?

Well, the first step is to know that confirmation bias is a thing, and a very powerful motivator.  We, our developers and our project managers are going to want us to "race to the finish", so there's a lot of pressure to "tick the box and say it's okay".

That statement I used is correct,

"I have some confidence that when I provide details similar to the situation described that the system should work as expected for the scenario provided"

So what's the solution?

Firstly, try different scenarios - make sure you cover all the scenarios you'd expect.

Secondly, vary the inputs to try a broad range of situations.

This was covered as I mentioned in my Back To Basics Testing series, but here we're looking at the "why".  Why we need to do this - the more we've covered the more confidence that statement gives.

But we can never be 100% confident - testing is always a trade-off between trying different inputs, expecting different outputs, whilst being aware all the time that the project clock is ticking, and we can't take forever to do this.

Good testing is about choosing those cases wisely, trying to cast a net across possibilities and experience to try and find potential problems.

Whatever happened to Pluto's craters?

Or ... a funny thing happened on the way to the Kuiper Belt.

I wrote my last article about Pluto in a flurry of excitement after we received the confirmation that New Horizons had made it past Pluto, and had captured data.  There was just a minor problem, to "do the science", the probe had to turn it's camera toward Pluto, and hence it's dish away from the Earth, and so it's only much after the event that we're starting to see some of the more closer images.

Even so, the latest images we'd had before it "went silent for science" he had back were pretty exciting.  I also said looking at them, "We see craters on Pluto, but not many of them".

In my last article, I talked about the planetary evolution science I learned in my degree.  We formed from a cloud of dust, it collides to make bigger and bigger objects (the Sun, it's planets and their satellites).  It's a somewhat chaotic process - eventually the planets form, but there's still asteroids and comets out there still colliding with them.  That can range from a fairly minor event (shooting star) to something more catastrophic (move over dinosaurs).

These impacts are most dramatically seen on the Moon and Mercury - because these world have no weather system, so remain preserved ...

We see craters on Venus, the Earth and Mars - but in each case the weather can erode these over time.  We also see them on many of the satellites of the ice giant planets, one of the most dramatic of which is Mimas, which has a giant crater which makes it look a little like the Death Star.  In actual fact we don't think you could have a bigger crater without the satellite splitting!

So we know that cratering happens throughout the universe - so look at that picture of Pluto.  Can you see those dark marks, especially the spots, which have to be craters?  Can you see the craters there?

There's just a problem which occured the day after I posted, and NASA scientist started posting, how one of the few things they haven't seen is any craters on Pluto.  But yet huge mountain ranges on such a small planet.  With so few craters out there either Pluto is relatively young, or something odd is going on out in the Kuiper belt, with incidents of cratering so incredibly low compared to elsewhere in the solar system.

So I got it wrong.  But what's been really interesting to me, is how I got it wrong.  It's a fantastic demonstration of an effect called confirmation bias, which I was always hoping to find a way to cover, and here we are!

First of all - I need you to be honest.  When I talked above about seeing dark rings in picture of Pluto, did you see craters too?

Confirmation bias works like this - if you look at a picture like that, which is slightly fuzzy, if you know that craters exist all throughout the solar system.  Then you are going to see shapes which your mind will recognise as craters.  Your mind is looking to confirm what it already thinks is there.  The problem is, there are no craters there, you've created a mirage of geology in your mind, you've fooled yourself.  And this in many ways is how much magic and illusion works.

It's a huge problem for us, because our eyes will sometimes see what our mind and heart really wants to see.  And this can be a problem for everyone, even scientists and testers.  How many times have you found a bug, and tried a few different options, and sure you know what the pattern is, then someone tries something around your patten, and shows a huge hole in it?  This is how it impacts us.

I'm going to talk a bit more about this in my next piece, focusing on how this impacts us in testing.  But for now I'll leave you with a picture of the Sputnik plain on Pluto, more "not craters" ...

Pluto: Because science ...

As discussed a little in my previous blog, the last few weeks have been really exciting to watch the New Horizon's probe get closer.

Let's start by showing you something - at the start of the year, this was as good as it got for Pluto - our image taken through the Hubble telescope ...

For all the amazing optics Hubble offers, it's not substitute for being up close and personal.  What has been amazing for me is to revisit much of the astronomy I learned whilst a graduate at the University Of Sheffield, and look at ways to apply the lectures of Professors Fred Combley and David Hughes to what I was seeing.  When we visited Neptune, I was just about to start studying with them, and hence I saw this planet only through their explanations.  But Pluto was mine to look at and make observations!

I'm also aware from discussions in my office, how very much I take what I know for granted, so this is an opportunity to give a quick lesson in science, to talk about how what we've seen in Pluto on the one hand confirms a lot of our models, but also asks so many new questions to a scientifically inquiring mind.

How our Solar System was formed

Actually to be honest, we don't know for certainty.  But what we have is a model which fits as many of the facts that we have - that essentially is what science is.

We believe that all star systems started out like this - a huge nebula of gas, which we see throughout the galaxy.

Typically these clouds are made of about 80% hydrogen, about 19% helium (the most abundant elements in the galaxy), and less than 1% is "everything else".

But gas clouds don't stay as gas clouds forever - even in the example above you can see that it's irregular, meaning some bits are more dense than others.  And this is where a phenomenon called accretion comes into play.  When the nebula is evenly distributed, gravitation (which is caused by the mass of an object) is pulling in all directions.  But once you have a clump of higher density, then this piece starts acting more like a point source, and starts to have higher gravitation.  This means it pulls in more gas from around it ... which means it gets heavier ... which means it's gravity increases ... which means it pulls in more material ...

So a lot of material from the nebula is pulled in, most goes to the star, but some actually accretes itself into other objects orbiting around the forming star.  This left over material forms the solar system as we know it.

All the planets in our solar system - and indeed our Sun - are made from the same source material.  So why do they seem so different?

One factor as we've talked about is gravitation from the Sun.  The Sun is the largest object in our solar system, and indeed the closer you'd have gone towards the Sun, the denser the cloud these objects would have formed from would be.

This presents us with a problem though - if this is true, then logically the cloud which would form Mercury would be the most dense, and therefore Mercury should be far and away the largest planet in the solar system (instead of Jupiter).  But it's not.

Clearly there's a second factor - which is the Sun's heat and it's solar wind.  When we look at spectral lines from the Sun, we have a good idea of the composition of the gas cloud we formed from.  The closer we are to the Sun, the hotter it is and the stronger the solar wind, especially once the Sun started becoming large enough to burn hydrogen and operate as what we know as a "main sequence star".

All this would mean the closer you were to the Sun, the more volatile lighter elements would be - especially gasses.  These would be blown outward by a combination of radiation from the sunlight and the solar wind (think much of how steam leaves a kettle).  What would be left behind is the heavier elements, the iron and silicates especially.  This is why planets like Mercury, Venus, Earth and Mars are called "the terrestrial planets" being made of mainly rocks and iron.

But as we've said, the further you get from the Sun, the colder it gets.  And at just beyond the asteroid belt, something odd starts to happen.  Gasses become that much less volatile and even "sticky", starting to clump together (and generally acting less like you'd expect a gas to behave).  When you combine "sticky gasses" with that phenomenon of greater density the closer you get to the Sun, you create a perfect storm, and something like Jupiter happens.

Jupiter is an oddity - it's 2.5 more massive than all the other planets of the solar system put together.  It's composed primarily though of hydrogen - the lightest element in the Universe.  And yet in Jupiter it's also the most dense planet.  The hydrogen of Jupiter thanks to the enormous gravitational pressure of the planet is acting in a way we just don't expect of hydrogen on Earth!

Temperature is everything - water and ice are the same substance, but one can be sculpted significantly more than the other.

As we move from Jupiter through the other "gas giants", we get the laws of diminishing returns, each planet smaller than the last (although Neptune and Uranus are almost on a parity).  Beyond Neptune the science says that the density of cloud that could form planets would never be able to form anything significant - or so our lectures in the 1990s went.

And they were right ... but just because we would not see anything significant, does not mean we'd see nothing.  Indeed, thanks to Hubble, we've started to detect a whole load of what we now call proto-planets which form what we now call the Kuiper belt.

This band seems analogous to the asteroid belt - but where that is made from terrestrial iron and silicates, we postulate that the Kuiper belt if made of frozen gasses and ices.  And much as Jupiter dominates the so called "gas giants", Pluto dominates this group of what we could call "ice dwarves".

Here comes New Horizons

Before we look at Pluto's pictures, let's talk about what we knew before this month,

• We knew that Pluto had several satellites, including Charon
• We had a good idea of Pluto's mass, thanks to the orbit with Charon
• We had an idea of the size of Pluto (although it's turned out to be a bit more massive than expected, but not by much)

Knowing it's mass and knowing it's size is a good start, as it allows us to work out it's density.  And knowing it's density we can make a good guess at it's composition just by comparing with other planets.  We expect from Pluto for it to be "mainly ice with some rock".

Okay - we've gone on enough - time to include a picture ...

This is a "false colour" picture of Pluto and Charon.  It's false colour because as Sunlight is about 0.2% that of at Earth, it's similar to trying to work out the colour of something via moonlight.  Thankfully the onboard Lorri camera is set for such conditions.

Let's start by talking to that colour - it's red, mainly red.  Having spent time with Professor David Hughes red makes me think rust and iron oxides.  More than likely, given the density, some kind of iron oxide within ice.

So what of the white heart-shaped band which has so captured the public's imagination?  Well it doesn't really fit.

To me, it's baffling.  I think we're looking at a mainly red world, with an ice patch.  I say think because I can see cratering on the red part of Pluto, which suggests it's older.  So where did this white patch come from?

It could be Pluto turning itself inside out, and the outflow of a massive ice volcano.  The cleaner white ice feels clearly younger than the red patches which dominate.  One way to be sure is to from radar modelling.  Whichever area has "lower altitude" is likely to be the elder area (on the Earth volcanic flow always goes over exiting flow).

What has fascinated me looking at Pluto and Charon next to each other ...

Pluto looks mainly red, with a white patch, and Charon mainly white with a red patch?  Is it possible that there was some kind of low-speed collision between a white Charon and a red Pluto, which would allow for transfer of material between the two?  I'm kind of reminded of a low speed "fender bender" bumping between two cars where paint from each car is transferred to the other.  Spectroscopic analysis over how similar these patches are on the two different worlds will either dispel or add fuel to this idea.

Likewise there's the mystery of the cratering.  We see craters on Pluto, but not many of them.  Our own Moon is heavily cratered, with some craters such as this star shape, the impact is extreme enough for material to almost reach the lunar escape velocity, with deposits over almost a quarter of the surface ...

We see similar phenomenon on many moons around the gas giants, but around such large, massive worlds which are gravitational magnets, that's not too surprising.

Pluto is about 20% the mass of the Moon, so any similar collisions should be as dramatic (if not more so), with material being blow around the globe.  But although we see cratering, so far we've not seen those tell-tall lines radiating from impacts.  Are impacts less frequent and of less energy being so far out?  Well, maybe more detailed images of the planet can answer that.

And then again there's the atmosphere.  The ability for a body to retain an atmosphere is proportional to it's mass - however although Plutos atmosphere is minuscule compared to Earth (no surprise there), it's significantly larger than the much more massive Moon, despite being much smaller in mass.

Again, one factor is believed to be temperature.  The Moon is (in cosmic terms) the same distance from the Sun as us (when it's facing the Sun).  Heat means energetic gas particles which would quickly leave the low gravity of the Moon.  Even though Pluto is much smaller, the gas particles there are less energetic, so less likely to be able to reach escape velocity.

Now add in Pluto's odd orbit, meaning at times it's closer to the Sun than others.  It's possible that some of it's ices could boil or "sublimate", venting gas which would become an atmosphere.  We see similar phenomenon in comets, which too are part-time residents of the Kuiper belt.

Have you noticed so far how every observation we've made so far leads to more questions, and things we can test and check?  This is the nature and fun of science - we answer questions, but in doing so propose new questions.  It's also one of the reasons I find testing to be such a fun discipline to be part of - good testing is good exploring, solving questions, asking new questions.

Stay tuned for more information from New Horizons as it's downloaded (it will take 16 months to download data from the probe) here.  And a special shout-out to friends Dan and Gabrielle who I've enjoyed sharing this journey with.

New Horizons - a final frontier

Part of me just does not want to sleep.  It's excitement.


Currently the New Horizons space probe is only a few days away from it's destined fly-by of Pluto.  This is quite a monumental day - it's likely to be the last time we get a "first" look at the surface of a new "planet" during my lifetime (technically it's designated a dwarf planet).  And already what we can see in grainy pictures is intriguing - this heart-shaped "continent" of what could be ice.

Pluto

It's exciting.  We know it's size and mass, it's about a fifth of our own Moon.  We have a good idea because of this and where it is that it's made of mainly ices - it's incredibly cold there, and the Sun has only 0.1% the heat we get in our orbit.

But we know nothing of what it looks like.  Pretty much we'd imagined some cratered object like our own Moon or Mercury, just made from a different material.  But as the image above shows, it looks like something else is going on, not just some uniform object which has been impacted with cosmic debris.  What kind of geological processes are in play here, and why does it's nearby partner dwarf-planet of Charon look so different?

Pluto and Charon

In the last few years we've found a handful of other dwarf-planets.  We used to think Pluto represented the edge of the solar system, but instead it marks the start of a whole other realm known as the Kuiper Belt where the dominance of gas giant planets like Jupiter, Saturn, Uranus and Neptune gives way to a cluster of much smaller objects - asteroids and dwarf planets.  It makes us appreciate how the solar system has no "edge" to it as we were once led to believe.

Most of all it's exciting because it's exploration.  We have never been here before.  We are seeing something totally new - we may go back (but with taking almost a decade to get there, it won't be any time soon), but if we do, it'll be to clarify things we learn in the next few days.  This is exploration at it's most raw - driven as all exploration is by curiosity.

You can find the latest here.

Seeking rapport in testing

I'm very aware that this blog has been a little quiet of late - some of which is intentional, and which I'm going to let you know about.

This year I've had a focus to my study - something I think could well take a few years to really come to fruition.  There are three sources I'm working through - although to me, I'm feeling there is a lot of overlap,

• I'm doing an online Cognitive Behaviour Therapy course
• I'm working through an excellent series of audio lectures on critical thinking
• I'm working through some of Jerry Weinberg's writing on the secrets of consulting

First of all - I do not want to become a therapist on any level.  But here's something I find, and really want to explore.

As a tester, I often find "unhealthy" attitudes towards testing, and I know I'm not alone.  What do I mean by unhealthy?  Well often it can be overly optimistic, not understanding what testing really does and the function it performs (blaming testers for the bugs they find sound familiar?).

This is supported by the exploration of psychology within the "Your Deceptive Mind" audio lectures.  We'd like to think the way we make decisions is we review the evidence, then decide a course of action.  In actual fact, we tend to emotionally decide an outcome in advance, then filter the evidence we encounter to support that outcome (a bit like our flat earther last year).

Likewise the models in our mind are always trying to find simple solutions to complex areas.  We don't like complexity.  Often politicians win at the ballot box because they have a simple (often woefully simple) solution to a problem, and many people feel they can get behind it.  Within testing this often manifests itself as "surely out there is a tool which will simplify all this for us", combined with the salesman patter of "this tool will reduce your need for testers".  It's something our brain wants to believe, and will often get us stung in the process.

In pretty much all this research a common term is coming across as a first line of addressing - rapport.  The word gets bandied about a lot - but what does it really mean?  Well to me it means giving someone room to explore and explain their approach and thought processes in a non-judgmental manner.

This seems to be the core of both counseling and consulting - if someone feels they're being judged, they tend to hold back, and especially be defensive regarding why they do things a certain way.  Our first step as a consultant is to understand the framework and the decisions which are being made.  A good consultant can then build on this, and suggest alternative methods of viewing things, sometimes nudging, sometimes challenging that world view.

Such change is slow, and can be a bit frustrating - but it also has the potential to be permanent.  You are finding ways to experiment and demonstrate factors to your clients to earn their faith and confidence.

It's a lot easier to enforce a new test doctrine or process, and have people adapt or leave.  But unless you've done some groundwork, people will either throw it away as soon as you leave, or (potentially worse) be slavishly devoted to your method without understanding it (and you've just created your very own Cargo Cult).

All this theory of course is wonderful - but as ever, it's the point where it's put into practice that's the true test.  That work - and the methods and approaches which can support this - is ongoing ...

If you have any thoughts or experience in this space, I'd of course love to hear about this in the comments section below.