Tuesday, June 14, 2016

AUTOMATION 5 - Unit Checks, Part 1: testing randomness

Warning - this example is going to contain some Java samples.  However, I don't want you to be put off that too much if you're not technical.  You should be able to follow the discussion and the outputs even if you're unfamiliar with code.

I love games - in fact so much so my son and I run a wargaming YouTube channel!

Most of the board/wargames we run rely on use dice to determine events - so a dice rolling application seems like something useful to build.

Below is a Java class I've written - don't worry if you can't read it, I'll explain the key features,

  • diceClass (int num) - it's called a constructor which creates a dice object with the number of sides (num) that I request
  • diceClass () - this is a default constructor.  If I don't specify a number, create a 6 sided dice, as they're the most common
  • rollDice() - this creates a random number as if the dice was rolled.  It can be any number between 1 and the number of sides the dice has.
  • returnDiceNumber() - I've used something called encapsulation here.  The number on the dice is private, which means it can't be accessed by anything outside of this code.  Instead I've provided a method which will return the currently rolled value on the dice!

import org.junit.Test; public class diceClass { private int numSides; // number of sides on the dice private int diceNumber; // the random number on the dice public diceClass (int num) { numSides = num; this.rollDice(); } public diceClass () { this(6); } public void rollDice() { diceNumber = 1 + (int)(Math.random() * numSides); } public int returnDiceNumber() { return diceNumber; } }

In layman's terms, I have code which will allow me to create a dice, roll it, find the value.  Pretty simple - but I still want to check it.  Because when I used to program using BASIC on the ZX Spectrum, I remember it was very easy to introduce a rounding error which would mean the first number would be less likely to be represented.

Right now I've not decided what checks to do, or even if it should be GUI or unit check.

Let's actually take a step back - if I was manually testing this what kind of test would I do?  Can I automate that?

Make sure every number comes up test

That seems pretty obvious - let's say the GUI for this looks like this ...

Hopefully it's pretty obvious - every time you hit "roll again" a new number comes up in the box, which currently says "4".

So what's a good test?  Well, I'd press "roll again" several times and expect a different result each time.  Actually random numbers don't work like that ... I'd expect a different number most times.

And I would expect to see the numbers 1-6 appear whilst doing it.

That is a really good first definition of a test.  However you're about to find the difference between a check and a test.  What you've just defined is far too ambiguous to become an automated check.

When we defined automation - it was a script driven set of controls for a system together with the determined outputs.  And that direction is missing.

As a manual tester, I'd run that until I was satisfied.  In an automated check, you have to define "satisfaction" as a hard number.

If I said "if I roll a 6-sided dice 6 times ... I expect to have seen the numbers 1, 2, 3, 4, 5 and 6 at least once" - that check is going to fail a lot.  Checks that fail a lot but don't find a true problem are bad, because they build up a lack of trust in the automation so that when something genuinely problematic occurs "we get fails all the time, but it doesn't matter".  But more on that later.

If I changed it to "if I roll a 6-sided dice 10 times ...  I expect to have seen the numbers 1, 2, 3, 4, 5 and 6 at least once", I expect to see the test fail less. But thanks to randomness, it's still possible to fail.

Let's actually define what that JUnit code would look like for a check with just 6 rolls

import org.junit.Test; import static org.junit.Assert.*; public class testDice { @Test public void everyNumberOccurs() { System.out.println("Confirming every number happens"); diceClass dice1 = new diceClass(); int numSides = 6; int[] countArray = new int[numSides]; int counter, index; int totalRolls = 6; //Roll the dice totalRolls times, count the results for(counter = 0; counter < totalRolls; counter++) { dice1.rollDice(); index = dice1.returnDiceNumber(); countArray[index-1]++; } //Rolling done, show us the numbers for(counter = 0; counter < numSides; counter++) { System.out.println("Assert: Number of " +
(counter+1) + " = " + countArray[counter]);

//Now confirm each is greater than 0
for(counter = 0; counter < numSides; counter++)
assertTrue("Did not find " +(counter+1)+ " rolled",
} }

Okay - let's run that.  What's neat about JUnit checks, they run as part of the build process for code, so you get a response pretty quick!

On our first go, you can see in the Eclipse console that not every number has occurred ...

Let's give it another go ... close but no cigar ...

And over under the Eclipse Failure Trace, the assertTrue has generated a failure,

Obviously I'm going to have to sample over a large enough set to reduce the chance of an error.  So I'm going to use as the basis of my check "if I roll a 6-sided dice 100 times ...  I expect to have seen the numbers 1, 2, 3, 4, 5 and 6 at least once".  That's probably excessive - but I've no real clue how many I'd do before I gave up in frustration.  And the automation needs an iron rule.

I'm going to change the value of totalRolls to 100 in the code above.  That means sampling 100 dice rolls.  Let's see what happens now ...

This check runs almost instantaneously.

Now of course I could use my GUI automation here to run this, it'd be okay - but much slower.  I'd need to wait for a press of the button, screen refresh,  read the value from the field, store it.  Generally I find it seems to take about 0.1 seconds for a screen refresh (hey, we get slow internet in New Zealand), unless you have a cut down webpage.  So checking 100 dice rolls could take a few seconds.

According to our iron rule ...

It probably makes sense to use a unit test over a GUI test for such a mass action.  The developer certainly finds out about the problem much simpler - it comes out as part of the build process.

More next time

I'm going to leave it here for today - we're going to look at more example checks we can do, and think a bit deeper of how this compares to GUI tests.  But for today we'll leave it here before you get overwhelmed.

Extension material

You might want to have a play around with this yourself - you might want to install Java Eclipse on your machine.  But don't forget you will also need to install the Java JDK as well from Oracle.

1 comment:

  1. Ass Wr Wb Saya ingin berbagi cerita kepada anda bahwa sy seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU LUSI yg dari singapura tentang AKI yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey 4D dr hasil ritual/ghaib dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.270.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI saya juga sudah bisa se sukses ini, pesan AKI yg slalu sy ingat setiap manusia bisa menjadi kaya, hanya saja terkadang mereka tidak tahu atau salah jalan, Banyak orang menganggap bahwa miskin dan kaya merupakan bagian dari takdir, Takdir macam apa? Tuhan tidak akan memberikan takdir yang buruk terhadap kita semua, cobaan yang Tuhan berikan merupakan pembuktian seberapa kuat Anda bertahan di dalamnya. Tuhan tidak akan merubah nasib Anda jika Anda tidak berusaha untuk merubahnya sendiri, Jadi teman2 yg dalam ke susahan jgn pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha. AKI MAULANA adalah guru spiritual terkenal di indonesia yg bisa melakukan ritual ghaib seperti:
    1.Pesugihan bank ghaib
    2.Ritual tembus togel/lottrey
    3.Transfer janin
    4.Pelaris usaha, jodoh DLL
    jika anda ingin mengubah nasib seperti saya silahkan KLIK DISINI PESUGIHAN DUNIA GHAIB


    I want to share a story to you that sy a migrant worker from malaysia and accidentally I open internet and I see the comment LUSI mom from singapore about AKI who has helped him become successful and finally I also tried to contact him and alhamdulillah he wanted Help me to give the number Togel / lottrey 4D dr ritual / unseen and alhamdulillah it really proved translucent and won RM.270.000 Ringgit, now I am back indon buying house and train even though sy just housemaid in selangor malaysia, sy very thankful To AKI MAULANA and do not forget to give thanks to ALLAH because through AKI I also can be this successful, AKI message which slalu sy remember every human being can become rich, only sometimes they do not know or wrong way, Many people assume that poor and rich Is a part of destiny, what kind of fate? God will not give a bad destiny to us all, God's temptation is proving how strongly you endure it.God will not change your fate if you do not try to change it yourself, So teman2 deep into tusah jgn never despair, when it's time god must be the road to where you want to try.AKI MAULANA is a famous spiritual teacher in Indonesia who can perform the occult rituals such as:
    1.Presugihan bank ghaib
    2.Ritual translucent togel / lottrey
    3.Transfer fetus
    4.Paris business, matching DLL
     If you want to change the fate Please visit the website CLICK HERE WORLD WIDE GHAIB