Monday, June 13, 2016

AUTOMATION 3: Automation as a service

To me, this section you're reading is probably THE most important section of the whole series.  So take a moment to let that sink in.

Before we're going to launch into a look at the technology, the how-to's and the tools, we need to take a step back.  Let's take a moment to think of automation as we would any IT deliverable, and start by working out who the users and customers are, and what they expect to get out of the system.

It's a radical way to look at it, but something a previous manager encouraged me to ponder.  When we think of automation our first thought is "which tool is best/everyone else using?", and from then on we just follow so many degrees of wrong.

What I was encouraged to do was to look at automation not as "which tool" but to think of it as a service.  To which the key question is who are the customers, and what do they want?  Who provides the service?

Indeed automation is in many ways a business process for IT companies, and for any such process, the tool is an important part, but it isn't all of it.

What evolved from this, are four key roles within the automation service, which we'll discuss below.

The Manager



I might be a test manager / project manager / business unit manager.
I am driven to keep everything as efficient as possible in terms of resources and cost.
I am not necessarily technical in background.

Driving needs for automation

  • I want to use automation so that I can reduce the number of hours my manual testers spend retesting.
  • I want to have to spend minimal resources on maintaining the automation going forward so that I’m the most efficient with the projects time and money.
  • I want to be able to see a high level status of the automation and when it’s been run so that I know if we have problems.

Questions you can expect from me,
  • But isn’t Selenium the same but free?
  • How much will it cost?  Is it a one off or ongoing cost?
  • What support is there?


The Developer



I am focused on creating new software.
I am not necessarily aware of how testing works, and may not be in charge of automating software.

Driving needs for automation
  • I want to get fast feedback on any new code I write so that the changes are still fresh in my head.
  • I want to be able to see which automated checks have failed, so that I know where I need to address problems.
  • I want to be able to look at a log file to understand if a failure is due to product or automation code so that I can be efficient in my work
  • I want to be able to look into log files and get enough information about any failure so that I can investigate and fix any issues.
  • I want to be able to run automation scripts so that I can check my software and find obvious failures
  • I want to be able to run subsets of automated scripts when needed to get fast feedback on fixes and builds

Questions you can expect from me,
  • What technologies does it work on?
  • What environments can it work from?

The Tester



I am focused on using my time to manually test our product to the best possible standard.
I am not necessarily technical.
I am not seeking to replace all manual testing with automation, but want to use automation to avoid any unnecessary and time consuming checking where possible.

Driving needs from automation
  • I want to be able to define new checks which I would like automated to reduce time consuming checking that I would otherwise have to perform again and again.
  • I want to implement checks that give me confidence that a feature is behaving in a consistent way as when I initially assessed it.
  • I want to be able to see at a high level all the checks that have been run, and their details so I know what the current status of the automation suit is.
  • I want to be able to look at any automated check and know what it does so that I can understand what the automation is doing, and hence focus my manual testing elsewhere.
  • I want to be able to look at an automated check log and understand that it’s doing so that any failures are meaningful to me.
  • I want to be able to run a wide range of automated regression checks so that I cover many areas of testing, to avoid unnecessary manual testing.

Questions you can expect from me
  • How do I define an automated check?
  • Where can I see my automated check?
  • Where can I see if my check passed?
  • How do I view the automation steps?


The Automator



It is my job to create and maintain the automation framework.
I might also work as a tester or a developer.

Driving needs from automation
  • I want to be able to write the code for automated check in a language I am familiar with so that I am comfortable with it, and don’t need additional training
  • I want to version control my automation script so I can revert to old versions as required
  • I want to be able to view the contents of automated checking logs so I can fix my scripts when things change
  • I want to be able to use a modular design in my automated scripts so that why my test code fails due to changes in the product design I can minimise the amount of rework needed in my software
  • I want to be able to run my automation scripts so that I know if my automation code works
  • I want to be able to look at a log file to understand if a failure is due to product or automation script so that I can be efficient in my work

Questions you can expect from me
  • What languages can it work in?




Using the four roles to help review your automation strategy

When setting up automation on a project, as we've discussed it's very easy to start with a trial, and just keep making scripts, and seeing how fast they run.

The four roles help you to review your overall process.  If you have a healthy process, everyone above will have their needs met, and be able to participate and benefit from it.

If however only one of those roles is really benefiting, to the determent of everyone else, then you have major problems, and you need to discuss it as a team, and find a way forward.

Some typical problems might be,

  • Only the automator can run automation scripts and view output.  You've created a bottleneck.
  • Automators create scripts without consultation with testers.  Testers hence have no idea what the automation regression covers, so have no strategy of working well with the automation.  Hence your hope of automation reducing what testing needs to cover goes out the window.
  • Automation is useful when it works.  But when it fails, you really need to be able to see why. 


6 comments:

  1. Hi Mike, very nice post.

    The roles you're describing are spot on. I reckon you could always add to the 'driving needs' and questions.
    For the manager one such driving need imo is "I want to raise the quality (sometimes without knowing what that is exactly) of our product or test effort by freeing up testers so they can test deeper.'

    For the tester one biggie I think is missing is 'I want to be able to extend my test coverage using automation tools to create tests that wouldn't be feasable manually'.

    Great post, thanks.

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Analogica Data is the Best Test Automation Services Company in India, offer software Automation Testing Services, Automation testing is nothing but enhancing efficiency, effectiveness and the coverage of your product.

    ReplyDelete
  4. Ass Wr Wb Saya ingin berbagi cerita kepada anda bahwa sy seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU LUSI yg dari singapura tentang AKI yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey 4D dr hasil ritual/ghaib dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.270.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI saya juga sudah bisa se sukses ini, pesan AKI yg slalu sy ingat setiap manusia bisa menjadi kaya, hanya saja terkadang mereka tidak tahu atau salah jalan, Banyak orang menganggap bahwa miskin dan kaya merupakan bagian dari takdir, Takdir macam apa? Tuhan tidak akan memberikan takdir yang buruk terhadap kita semua, cobaan yang Tuhan berikan merupakan pembuktian seberapa kuat Anda bertahan di dalamnya. Tuhan tidak akan merubah nasib Anda jika Anda tidak berusaha untuk merubahnya sendiri, Jadi teman2 yg dalam ke susahan jgn pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha. AKI MAULANA adalah guru spiritual terkenal di indonesia yg bisa melakukan ritual ghaib seperti:
    1.Pesugihan bank ghaib
    2.Ritual tembus togel/lottrey
    3.Transfer janin
    4.Pelaris usaha, jodoh DLL
    jika anda ingin mengubah nasib seperti saya silahkan KLIK DISINI PESUGIHAN DUNIA GHAIB



    THE STORY OF SUCCESS MOTHER RATNAH PUSPITA

    I want to share a story to you that sy a migrant worker from malaysia and accidentally I open internet and I see the comment LUSI mom from singapore about AKI who has helped him become successful and finally I also tried to contact him and alhamdulillah he wanted Help me to give the number Togel / lottrey 4D dr ritual / unseen and alhamdulillah it really proved translucent and won RM.270.000 Ringgit, now I am back indon buying house and train even though sy just housemaid in selangor malaysia, sy very thankful To AKI MAULANA and do not forget to give thanks to ALLAH because through AKI I also can be this successful, AKI message which slalu sy remember every human being can become rich, only sometimes they do not know or wrong way, Many people assume that poor and rich Is a part of destiny, what kind of fate? God will not give a bad destiny to us all, God's temptation is proving how strongly you endure it.God will not change your fate if you do not try to change it yourself, So teman2 deep into tusah jgn never despair, when it's time god must be the road to where you want to try.AKI MAULANA is a famous spiritual teacher in Indonesia who can perform the occult rituals such as:
    1.Presugihan bank ghaib
    2.Ritual translucent togel / lottrey
    3.Transfer fetus
    4.Paris business, matching DLL
     If you want to change the fate Please visit the website CLICK HERE WORLD WIDE GHAIB




    ReplyDelete
  5. Nicely explained. Here you described the well written article from your in-depth knowledge. Truly impressive and nice information

    Java Training in Chennai Core Java Training in Chennai Core Java Training in Chennai

    Java Online Training Java Online Training JavaEE Training in Chennai Java EE Training in Chennai

    ReplyDelete