Monday, August 24, 2015

Playing office games ...

This weekend, my son played what was to be his last soccer game for his school - yes, next year he will (with luck) be off to University!

It was a hard game, and his team lost.  It was made more difficult by the attitude of the opposition - they'd previously been found cheating in the season (fielding a lot of players who weren't even members of the school).

In this game, Cameron faced a team who seemed to play dirty at every opportunity - some brutal tackles (which would be banned even in rugby), a tendency to be forever calling for a penalty, and just being rude at every opportunity.  Cameron as goalkeeper played the game of his life keeping balls out of the goal, but it wasn't quite enough.

Cameron called his team in at the end, and called for them to cheer and applaud the opposition.  Then privately, he gave his own opinion,

"I'm actually glad we lost, if winning would have involved behaving like they did".

A double dose of sarcasm for sure, but surprisingly short on bitterness.  What I love about my son is that he absolutely meant it.

Of course, his words have been ringing around my head ever since - how did a teenager get to act so mature about something like this?

It made me reflect back about a team I used to work on, many years ago back in the UK.  I worked in a large multinational IT company, the kind they write Dilbert about.  A lot of aspects of the company were not too bad, then in 2003 I was assigned to a particularly toxic project I'll called NavyNet.

This project had a lot of problems (ironically which could all be traced back to people problems).  For me the most obvious was the kind of "snitch culture" that went on.  You never got feedback directly, but there seemed to about a group (which I've always assumed to be a minority) who seemed to go around reporting/complaining about coworkers regularly.

A few of us seemed to spend every month being told that "someone has complained about you".  And having to go through it with a HR representative.  Ironically this caused some deep friendships, as I soon found it wasn't just me.  It was really frustrating, I was even told by HR that "the problem isn't so much your work ... it's the perception about your work".  What does that even mean?

A bit like my son, I had a difficult game to decide how to play.  I have to admit I gave serious thought to joining the accusers and trying to get the drop on someone before they got the drop on me.  But instead I tried to go "I'll show them", and worked harder, and tried to showcase what I was doing more.  But the complaints just kept coming.

Then after two years, I learned how to win.  I left.  And I never went back.

I found other projects and companies who took the office environment and culture more seriously.  I kept in touch with my friends on the project, and they were more miserable than ever.  They seemed truly envious of the fact that I had moved on, and was happy now.  But it had to come with a very hard realisation - I could never make this project happy.

It was a hard lesson.  Unlike my son at his game, I have to admit, I do feel bitter - I pretty much view those as two completely wasted years of my life.  Years that I will not get back.

Talking during this years Kiwi Workshop on Software Testing (KWST), Chris Priest had an experience report on such toxic work environments which blew me away.  You could tell those who'd experienced such offices, especially by their passion to avoid such places in future.

For a time in the late 90s/ early 00s, it felt like this kind of Office Space experience was just going to become more and more frequent.  But thankfully at KWST, there was a good proportion of attendees (typically the younger testers) who had never experienced that kind of environment in their careers - and who I'll admit to having a degree of jealousy over that.

So there is hope.  But as I've said to someone at KWST - we're really good at talking up our successes.  What's hidden is someone successful typically has built this on a lot of hidden failures and hard times, which lie beneath the water line.  This why Chris Priest's experience report was so powerful.

Like my son's game and the NavyNet project, if success in a field involves compromising your fundamental sense of fairness.  If success involves doing something which you feel deeply makes you unhappy and respect yourself less, then maybe the only way to win is to smile, and walk away.

Saturday, August 1, 2015

Windows 10 - playtime ...

My Windows 8 machine asked me if I wanted to update to Windows 10 today, I didn't hesitate.  Alas, as soon as I did, I remembered how I have a testing event in 2 days time, and I kind of want my machine to be fully working for it.

Ideally for most upgrades like this, you really want to wait a few weeks for it to be released and them to "work the kinks out".  But we're testers right?  Even if it's inconvenient, you just want to know, get in there and look and play.

I've seen a few complaints about Windows 10 and comparability on my friends updates on Facebook.  I was a bit baffled by that - because these are the same friends who tell me they buy Apple laptops because "Bill Gates".  [So how would they know?]

Maybe I'm just a product of the 90s, I still find FRIENDS pretty funny, and think listening to Blur is pretty hip.  But I think Windows is an amazing product overall, with the sheer spectrum of machines it will end up running on.  Recent changes for Windows 8 I found difficult at first, but then I really fell in love with it when I looked back.

Most of all, as a tester, I like to play around with something new.  And I could easily lose a whole weekend just trying things out.  So far I've found that my anti-virus software needed to be uninstalled and reinstalled (well, techinically the platform has just changed underneath it).  I'm also finding my Google Chrome crashed - and wondering if it'll happen again.  [Stand by not to be shocked, but upgrading to Windows 10 resets your browser preference to Microsoft IE ... I mean Edge.  Nice try guys, but ....]

I've also had a go with Cortana, particularly trying to see what it can do, and how good the voice recognition is on it.  This is me trying to say "Hal 9000" ...

Again, probably not surprising, but Cortana links to the Bing search engine.

At the end of the day, I think releases like this are an exciting opportunity to explore, and get a look at a new system.  In our work days, this is what we live for - to discover the good and the bad in a new piece of software.  And most of all to work through the problems.

Enjoy ...

Friday, July 31, 2015

The lion value system ...

When I was working reviewing More Agile Testing for Janet Gregory and Lisa Crispin, there was a whole forum of discussion which was running parallel to the book creation.

One topic we explored though was getting the balance right.  Everyone knows one of the great things about agile teams is that they interact and they talk about issues as they come up.  They're highly social.

However at the same time, there are stretches where people need to focus, and be uninterrupted for a time.  How can you get the balance right?

Back at Kiwibank we had an interesting system - our team had worked hard to think about what our "tribal identity" was.  In the end we thought of a lion as a mascot - because they represented the courage, ownership and attitude which we felt was core to what we wanted to achieve as a team.

Each of us had a little plush lion on our desk.  When you needed to be left alone, he was placed strategically to tell the rest of the team (although new people would have to be told this rule quite quickly as they'd be forever asking "what's with the cute lion" to the person who expected to be left alone).

People weren't allowed to have this lion on display for longer than half a day without explaining - it was there to be used as needs be.

Likewise if we had a meeting, and there was something you needed someone to know was really important about, and you might need them to listen to (especially if you weren't the loudest voice in the room), you'd bring the lion with you to the meeting.

In a way, the lion represented a kind of social contract between team members, and it's something I'm looking back at and wondering if it's worth (and fun) implementing on my new team ...

Thursday, July 30, 2015

Here Be Dragons - Critical Thinking film

We've been talking about critical thinking a lot in part posts.  I am actually hoping this year to explore aspects of critical thinking in more detail, especially looking at how critical thinking impacts testing.

I was introduced to the term "critical thinking" really in the RST course run by James Bach.  He has a lot of areas he's explored critical thinking for testing, and I'd like to build on that - or at least find my own way of describing the process.

Until then, I want to point you in the direction of this video "Here Be Dragons" which explores in a short 40 minute form some of the fundamental concepts of critical thinking.

Enjoy ...

Monday, July 20, 2015

Confirmation bias in testing ...

In my previous post I explored about some mistakes I made looking at the New Horizons mission to Pluto, introducing the concept of confirmation bias.

Like many people, although I was vaguely aware of the concept, it took James Bach's RST course to formally introduce me to the concept.  As I discussed in the previous article, our eyes will sometimes see what our mind and emotions want us to see.

In fact it could be said that the very profession of testing occurs because of this effect.

Take a developer who is writing a registration page for a new system.  He goes through it, provides his name, email, date of birth, and it taken to a "Welcome Stuart Cook!" message.  He concludes "it works for me" and moves onto his next exciting coding task.

Can we really say at this point that registration has been "tested"?  The developer enters some data, he sees what he want's to see - he gets a "welcome" message - and to him this confirms that everything is working.

We have all occasionally fallen into this trap (I know I have) - one test does not cover everything.  Being aware that we have a tendency through confirmation to jump to "that works" is a key part of the RST course.

Entering some details, and getting "Welcome Stuart" is a good start in testing.  But we're not there yet.

Is it for instance just a message?  How do we know for sure that account has been created?  We could check the database, and maybe do a login, to confirm we can use the account.  If Stuart hasn't made the login page yet, then maybe we will have to settle for checking the database.  You were going to check the database weren't you?

A minor thing - but what if we create an account for someone who isn't called Stuart - would we still get "Welcome Stuart" as a message?

Then there's the case of "what if we provide really long, diverse fields for answers to the registration questions - what happens then?", plus of course "what if I provide junk answer to the registration page - does it still make my account?".

All these many heuristic ways of testing a registration page were covered back in my Back To Basics series, which is worth a recap.

Sometimes as testers we're seen as being a bit tricky and difficult to work with.

When Stuart the developer checks his name in the scenario, he says "it works" and moves on.

But when we as testers do the same test we say (or rather we should say) "I have some confidence that when I provide details similar to the situation described that the system should work as expected for the scenario provided".

Isn't that written in a lot of safety language?  Some in software development feel uncomfortable.  Lets take some time to explore the phrase I used, and why I used it ...

"when I provide details similar to the situation described"

I provided the following details - they include first/middle/last name of a certain length using certain characters, a certain date of birth, and a certain password complexity.  If the system is presented with data which includes,

  • Characters I didn't include
  • Strings which are much longer/shorter
  • Date of birth which is significantly different to that I used
  • Password which radically differs in structure

Then I will be less confident of the outcome of the system, as these inputs differ considerably from the scenario that I used.

Are you comfortable with that?

"the system should work for the scenario provided"

If I've tested the "account creation" of a registered user, I cannot then say I've also tested the "account rejected" scenario.  This is pretty basic, but again in our confirmation bias, if we see an account has been created we can jump to "well this works", where there are really some scenarios that you really DO NOT want it to work.

"I have some confidence"

What a weird statement to start with?  Surely you're sure aren't you?  Okay - I take the above points, but you've created a Stuart Cook user - surely you're 100% confident that if provided with the same information, that you can create a Stuart Cook account?

Actually no - indeed if you've already provided details for a Stuart Cook account, then surely providing the same identical details will mean an error - you can't create two identical accounts after all.  And if you vary the inputs, then there is a level (however minor) of uncertainty.

Okay - so let's say we delete the Stuart Cook account after you've made it.  Surely you're 100% confident you can make it again?

Nope.  One thing I've learned recently since working on highly available test rigs with load balancers.  Sometimes you can get the most frustrating kinds of errors.  You try something and it fails - so you get a developer over, and run it 4 times, each time it passes.  Your developer walks away in disgust, and the problem comes back.

Rather than being transferred to the tester funny farm, it turns out that the problem you're experiencing happens only on one server, and just by pure luck, you've been pointing to the wrong server every time you've been on your own testing!

So what's the cure?

Well, the first step is to know that confirmation bias is a thing, and a very powerful motivator.  We, our developers and our project managers are going to want us to "race to the finish", so there's a lot of pressure to "tick the box and say it's okay".

That statement I used is correct,

"I have some confidence that when I provide details similar to the situation described that the system should work as expected for the scenario provided"

So what's the solution?

Firstly, try different scenarios - make sure you cover all the scenarios you'd expect.

Secondly, vary the inputs to try a broad range of situations.

This was covered as I mentioned in my Back To Basics Testing series, but here we're looking at the "why".  Why we need to do this - the more we've covered the more confidence that statement gives.

But we can never be 100% confident - testing is always a trade-off between trying different inputs, expecting different outputs, whilst being aware all the time that the project clock is ticking, and we can't take forever to do this.

Good testing is about choosing those cases wisely, trying to cast a net across possibilities and experience to try and find potential problems.

Whatever happened to Pluto's craters?

Or ... a funny thing happened on the way to the Kuiper Belt.

I wrote my last article about Pluto in a flurry of excitement after we received the confirmation that New Horizons had made it past Pluto, and had captured data.  There was just a minor problem, to "do the science", the probe had to turn it's camera toward Pluto, and hence it's dish away from the Earth, and so it's only much after the event that we're starting to see some of the more closer images.

Even so, the latest images we'd had before it "went silent for science" he had back were pretty exciting.  I also said looking at them, "We see craters on Pluto, but not many of them".

In my last article, I talked about the planetary evolution science I learned in my degree.  We formed from a cloud of dust, it collides to make bigger and bigger objects (the Sun, it's planets and their satellites).  It's a somewhat chaotic process - eventually the planets form, but there's still asteroids and comets out there still colliding with them.  That can range from a fairly minor event (shooting star) to something more catastrophic (move over dinosaurs).

These impacts are most dramatically seen on the Moon and Mercury - because these world have no weather system, so remain preserved ...

We see craters on Venus, the Earth and Mars - but in each case the weather can erode these over time.  We also see them on many of the satellites of the ice giant planets, one of the most dramatic of which is Mimas, which has a giant crater which makes it look a little like the Death Star.  In actual fact we don't think you could have a bigger crater without the satellite splitting!

So we know that cratering happens throughout the universe - so look at that picture of Pluto.  Can you see those dark marks, especially the spots, which have to be craters?  Can you see the craters there?

There's just a problem which occured the day after I posted, and NASA scientist started posting, how one of the few things they haven't seen is any craters on Pluto.  But yet huge mountain ranges on such a small planet.  With so few craters out there either Pluto is relatively young, or something odd is going on out in the Kuiper belt, with incidents of cratering so incredibly low compared to elsewhere in the solar system.

So I got it wrong.  But what's been really interesting to me, is how I got it wrong.  It's a fantastic demonstration of an effect called confirmation bias, which I was always hoping to find a way to cover, and here we are!

First of all - I need you to be honest.  When I talked above about seeing dark rings in picture of Pluto, did you see craters too?

Confirmation bias works like this - if you look at a picture like that, which is slightly fuzzy, if you know that craters exist all throughout the solar system.  Then you are going to see shapes which your mind will recognise as craters.  Your mind is looking to confirm what it already thinks is there.  The problem is, there are no craters there, you've created a mirage of geology in your mind, you've fooled yourself.  And this in many ways is how much magic and illusion works.

It's a huge problem for us, because our eyes will sometimes see what our mind and heart really wants to see.  And this can be a problem for everyone, even scientists and testers.  How many times have you found a bug, and tried a few different options, and sure you know what the pattern is, then someone tries something around your patten, and shows a huge hole in it?  This is how it impacts us.

I'm going to talk a bit more about this in my next piece, focusing on how this impacts us in testing.  But for now I'll leave you with a picture of the Sputnik plain on Pluto, more "not craters" ...

Wednesday, July 15, 2015

Pluto: Because science ...

As discussed a little in my previous blog, the last few weeks have been really exciting to watch the New Horizon's probe get closer.

Let's start by showing you something - at the start of the year, this was as good as it got for Pluto - our image taken through the Hubble telescope ...

For all the amazing optics Hubble offers, it's not substitute for being up close and personal.  What has been amazing for me is to revisit much of the astronomy I learned whilst a graduate at the University Of Sheffield, and look at ways to apply the lectures of Professors Fred Combley and David Hughes to what I was seeing.  When we visited Neptune, I was just about to start studying with them, and hence I saw this planet only through their explanations.  But Pluto was mine to look at and make observations!

I'm also aware from discussions in my office, how very much I take what I know for granted, so this is an opportunity to give a quick lesson in science, to talk about how what we've seen in Pluto on the one hand confirms a lot of our models, but also asks so many new questions to a scientifically inquiring mind.

How our Solar System was formed

Actually to be honest, we don't know for certainty.  But what we have is a model which fits as many of the facts that we have - that essentially is what science is.

We believe that all star systems started out like this - a huge nebula of gas, which we see throughout the galaxy.

Typically these clouds are made of about 80% hydrogen, about 19% helium (the most abundant elements in the galaxy), and less than 1% is "everything else".

But gas clouds don't stay as gas clouds forever - even in the example above you can see that it's irregular, meaning some bits are more dense than others.  And this is where a phenomenon called accretion comes into play.  When the nebula is evenly distributed, gravitation (which is caused by the mass of an object) is pulling in all directions.  But once you have a clump of higher density, then this piece starts acting more like a point source, and starts to have higher gravitation.  This means it pulls in more gas from around it ... which means it gets heavier ... which means it's gravity increases ... which means it pulls in more material ...

So a lot of material from the nebula is pulled in, most goes to the star, but some actually accretes itself into other objects orbiting around the forming star.  This left over material forms the solar system as we know it.

All the planets in our solar system - and indeed our Sun - are made from the same source material.  So why do they seem so different?

One factor as we've talked about is gravitation from the Sun.  The Sun is the largest object in our solar system, and indeed the closer you'd have gone towards the Sun, the denser the cloud these objects would have formed from would be.

This presents us with a problem though - if this is true, then logically the cloud which would form Mercury would be the most dense, and therefore Mercury should be far and away the largest planet in the solar system (instead of Jupiter).  But it's not.

Clearly there's a second factor - which is the Sun's heat and it's solar wind.  When we look at spectral lines from the Sun, we have a good idea of the composition of the gas cloud we formed from.  The closer we are to the Sun, the hotter it is and the stronger the solar wind, especially once the Sun started becoming large enough to burn hydrogen and operate as what we know as a "main sequence star".

All this would mean the closer you were to the Sun, the more volatile lighter elements would be - especially gasses.  These would be blown outward by a combination of radiation from the sunlight and the solar wind (think much of how steam leaves a kettle).  What would be left behind is the heavier elements, the iron and silicates especially.  This is why planets like Mercury, Venus, Earth and Mars are called "the terrestrial planets" being made of mainly rocks and iron.

But as we've said, the further you get from the Sun, the colder it gets.  And at just beyond the asteroid belt, something odd starts to happen.  Gasses become that much less volatile and even "sticky", starting to clump together (and generally acting less like you'd expect a gas to behave).  When you combine "sticky gasses" with that phenomenon of greater density the closer you get to the Sun, you create a perfect storm, and something like Jupiter happens.

Jupiter is an oddity - it's 2.5 more massive than all the other planets of the solar system put together.  It's composed primarily though of hydrogen - the lightest element in the Universe.  And yet in Jupiter it's also the most dense planet.  The hydrogen of Jupiter thanks to the enormous gravitational pressure of the planet is acting in a way we just don't expect of hydrogen on Earth!

Temperature is everything - water and ice are the same substance, but one can be sculpted significantly more than the other.

As we move from Jupiter through the other "gas giants", we get the laws of diminishing returns, each planet smaller than the last (although Neptune and Uranus are almost on a parity).  Beyond Neptune the science says that the density of cloud that could form planets would never be able to form anything significant - or so our lectures in the 1990s went.

And they were right ... but just because we would not see anything significant, does not mean we'd see nothing.  Indeed, thanks to Hubble, we've started to detect a whole load of what we now call proto-planets which form what we now call the Kuiper belt.

This band seems analogous to the asteroid belt - but where that is made from terrestrial iron and silicates, we postulate that the Kuiper belt if made of frozen gasses and ices.  And much as Jupiter dominates the so called "gas giants", Pluto dominates this group of what we could call "ice dwarves".

Here comes New Horizons

Before we look at Pluto's pictures, let's talk about what we knew before this month,

  • We knew that Pluto had several satellites, including Charon
  • We had a good idea of Pluto's mass, thanks to the orbit with Charon
  • We had an idea of the size of Pluto (although it's turned out to be a bit more massive than expected, but not by much)

Knowing it's mass and knowing it's size is a good start, as it allows us to work out it's density.  And knowing it's density we can make a good guess at it's composition just by comparing with other planets.  We expect from Pluto for it to be "mainly ice with some rock".

Okay - we've gone on enough - time to include a picture ...

This is a "false colour" picture of Pluto and Charon.  It's false colour because as Sunlight is about 0.2% that of at Earth, it's similar to trying to work out the colour of something via moonlight.  Thankfully the onboard Lorri camera is set for such conditions.

Let's start by talking to that colour - it's red, mainly red.  Having spent time with Professor David Hughes red makes me think rust and iron oxides.  More than likely, given the density, some kind of iron oxide within ice.

So what of the white heart-shaped band which has so captured the public's imagination?  Well it doesn't really fit.

To me, it's baffling.  I think we're looking at a mainly red world, with an ice patch.  I say think because I can see cratering on the red part of Pluto, which suggests it's older.  So where did this white patch come from?

It could be Pluto turning itself inside out, and the outflow of a massive ice volcano.  The cleaner white ice feels clearly younger than the red patches which dominate.  One way to be sure is to from radar modelling.  Whichever area has "lower altitude" is likely to be the elder area (on the Earth volcanic flow always goes over exiting flow).

What has fascinated me looking at Pluto and Charon next to each other ...

Pluto looks mainly red, with a white patch, and Charon mainly white with a red patch?  Is it possible that there was some kind of low-speed collision between a white Charon and a red Pluto, which would allow for transfer of material between the two?  I'm kind of reminded of a low speed "fender bender" bumping between two cars where paint from each car is transferred to the other.  Spectroscopic analysis over how similar these patches are on the two different worlds will either dispel or add fuel to this idea.

Likewise there's the mystery of the cratering.  We see craters on Pluto, but not many of them.  Our own Moon is heavily cratered, with some craters such as this star shape, the impact is extreme enough for material to almost reach the lunar escape velocity, with deposits over almost a quarter of the surface ...

We see similar phenomenon on many moons around the gas giants, but around such large, massive worlds which are gravitational magnets, that's not too surprising.

Pluto is about 20% the mass of the Moon, so any similar collisions should be as dramatic (if not more so), with material being blow around the globe.  But although we see cratering, so far we've not seen those tell-tall lines radiating from impacts.  Are impacts less frequent and of less energy being so far out?  Well, maybe more detailed images of the planet can answer that.

And then again there's the atmosphere.  The ability for a body to retain an atmosphere is proportional to it's mass - however although Plutos atmosphere is minuscule compared to Earth (no surprise there), it's significantly larger than the much more massive Moon, despite being much smaller in mass.

Again, one factor is believed to be temperature.  The Moon is (in cosmic terms) the same distance from the Sun as us (when it's facing the Sun).  Heat means energetic gas particles which would quickly leave the low gravity of the Moon.  Even though Pluto is much smaller, the gas particles there are less energetic, so less likely to be able to reach escape velocity.

Now add in Pluto's odd orbit, meaning at times it's closer to the Sun than others.  It's possible that some of it's ices could boil or "sublimate", venting gas which would become an atmosphere.  We see similar phenomenon in comets, which too are part-time residents of the Kuiper belt.

Have you noticed so far how every observation we've made so far leads to more questions, and things we can test and check?  This is the nature and fun of science - we answer questions, but in doing so propose new questions.  It's also one of the reasons I find testing to be such a fun discipline to be part of - good testing is good exploring, solving questions, asking new questions.

Stay tuned for more information from New Horizons as it's downloaded (it will take 16 months to download data from the probe) here.  And a special shout-out to friends Dan and Gabrielle who I've enjoyed sharing this journey with.