Friday, January 27, 2017

AUTOMATION 27 - Goals, constraints, strategy

Technical level: **

I originally posted my planned curriculum here back in June.  It was based on my understanding of automation at the time.

I've used the series and the interest it's generated to engage with a lot of testers.  I've spoken with a lot of automators around Wellington, and when I've been in conference.  I've collected all their stories, sifted them, and clustered the points they've raised.

A lot of what they've said has been nice affirmations of what I already knew.  But truth is, we engage with others not only to confirm our biases, but hopefully to learn and to stretch ourselves.  Today we're going to revisit some material which probably should go right at the beginning.  Most of it doesn't undermine what I've talked about before, but it does expand on it nicely.


I talked about this previously here under "automation as a service", but I'm shifting focus a little to really think more firmly about goals.

We're used to people coming up to us when talking about automation and saying "oh my god, there is a great tool, a fantastic tool".  We tend to start with this magic tool that everyone says is so easy to use.

In all my conversations last year I noticed a common theme about people who had stories to tell me.  They were all what I've come to call "second generation automators".

They had a common tale which went something like "we started using this system ... it was record and playback because our testers didn't know how to code".  The rest of their story played out pretty much as one of my denial case studies - they'd invested heavily in the tool, but every time the code changed, all the tests broke, and the maintenance for that was painful.

Inevitably what happened is they had to bin what they'd done to date and start again.  But second time around despite being under incredible pressure to get underway, many of they used the time to plan out much more.

To do this many reflected on their goals.  A common theme of those goals were that they were trying to speed up the feedback process.  If a new build came out, they wanted to find the obvious problems quickly.  These tended to be around making sure they could complete an end-to-end transaction, and that key activities could be completed.

There were differences of opinion about whether the automation was there to evaluate a new build vs doing a deep regression test.  And we'll talk about that more precisely in a later article.

The two key constraints of automation

From their previous failed attempts at automation, there were two key constraints which groups had come to appreciate,
  • Automation takes time to do well
  • Automation needs some training/expertise to make maintainable
 These were immovable written-in-stone rules, for which they had to build strategies to cope with.  We'll talk more about them below ...

Automation takes time to do well

That record-and-playback tool they'd used before, that could write a script as you tested?  They'd got burned there because every time they're try to rerun it, it'd need a tweak or ten.

Generally you could create an automated script full of checks in the time it'd take you to run about a dozen manually.  If you were going to automate something you had to be sure it was something that had value in checking repeatedly.

Every group used what I call "the strategy of focus" to deal with this constraint.  They knew their automation was going to take a while - they'd never probably automate everything.  What they needed it to do was to check (a) critical elements and (b) anything that too a long time to do manually.

I had a great chat with an automation guru from TradeMe who had this great insight.  If you're working especially on an agile team, you always are working from your most critical elements to your least critical.  It can be a trap for testers to work on new features, and want to automate them.  But in truth the most critical features to your system are those you've already built!

Every group tried in some way to accept that the automation would be an ongoing concern.  They key solution was to prioritise and focus.  They all tried to involve testers in capturing ideas for automated scripts.  One of my favourite tales was of a group which kept a spreadsheet where testers could contribute ideas for script as they came to them, and the whole team would vote to prioritise what to build next.  Democracy in action!

And don't fall into my key trap.  I often find myself going that "everyone" means developers and testers.  Sure testers are a great source of ideas for automation scenarios - we excel at this.  But don't forget particularly to get business people in to help prioritise and come up with ideas.  Occasionally they'll say "this happened once in production", and the thing that happened really scared them, they'd like to know everything has been done to reduce the risk of it ever happening again.  That's the kind of thing that's worth doing.  Not necessarily every bug that made it to production, but the ones that gave the business the jitters.

Automation needs some training/expertise to make maintainable

Again, this came from the sting of "anyone can create scripts using our record-and-playback".  That had got groups into a mess.

There was an embracing of the fact that to write automation, you needed some kind of training and some kinds of coding skills.  But also you didn't want to pick up a tool for which you needed specialist bespoke training - what I like to call "Hogwarts for automation".

Generally there's a desire to use a commonly available language.  If you're a software shop that develops in Java, it probably makes a lot of sense to develop your automation in Java.  You might have some people around who not only can help, but are really good at creating and designing maintainable Java code (you're really hope).

[Side story, once for a customer I had to learn to problem in Lisp to use their automation.  A truly bizarre language, and one of the oldest programming languages still used]

Selenium Webdriver with Java is a really popular combination.  It was one of the reasons that I spun off my Java series.  But it's important to remember this is just a handy GUI automation, as we've talked about previously, finding room for good unit and API testing is really crucial (and we'll talk about this later).

Indeed this was an important takeaway.  Many groups had been lured into their first attempt at automation through the sales pitch of "a tool".  But they'd come to appreciate that automation was not a tool, but a toolbox.  Much like when an engineer comes to your house, she doesn't just bring one spanner, she brings a toolbox!

Unit testing, API testing, units testing are different tools in our automation toolbox, and it's important we learn to use the right one for the right job.  Which leads me to my next post ...

Want to know more?

I'm talking at the upcoming Unicom automation conference in Wellington.  I will be speaking about some of what I've learned not only from my experience, but through going out and talking to people about their automation strategies.  There'll be some other great speakers there as well!


  1. Saya ingin berbagi cerita kepada anda bahwa saya IBU HERAWATI seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU DARNA yg dari singapur tentan AKI SYHE MAULANA yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey toto 4D dr hasil ritual/ghaib melalui jarak jauh dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.230.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy Cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI SYHE MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI MAULANA saya juga sudah bisa sesukses ini. Jadi kawan2 yg dalam kesusahan jg pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha, ini adalah kisah nyata dari seorang TKW, AKI MAULANA adalah guru spiritual terkenal di indonesia. jika anda ingin seperti saya kunjungi situs/website KLIK DISINI RITUAL PESUGIHAN DUNIA GHAIB


    I want to share the story to you that I am HERAWATI MOTHER a TKW from malaysia and accidentally I open internet and I see DARNA MOTHER comment from singapur about AKI SYHE MAULANA who has helped him become successful and finally I also tried to contact him and alhamdulillah he Want to help me to give the number Togel/lottrey toto 4D dr ritual results / unseen and alhamdulillah it really proved translucent and won RM.230.000 Ringgit, now I am back indon buying house and train although sy Just a housemaid at selangor malaysia, sy very thankful To AKI SYHE MAULANA and do not forget to give thanks to ALLAH because through AKI MAULANA I also can be as successful as this.So kawan2 who in distress jg ever break down, when it's time god must be the road of origin you want to try, this is the true story of a migrant worker, AKI MAULANA is a famous spiritual teacher in Indonesia. If you want to like me visit the website CLICK HERE RITUAL GHAIB TEMBUS LOTTREY

  2. Nicely explained. Here you described the well written article from your in-depth knowledge. Truly impressive and nice information

    Java Training in Chennai Core Java Training in Chennai Core Java Training in Chennai

    Java Online Training Java Online Training JavaEE Training in Chennai Java EE Training in Chennai

  3. Hi Dear!!!
    This Is My First Post.
    We Are a Web Service Provider,
    Online Casino Play Baccarat Online Casino Via Mobile Phone, IPhone, IPad and Roulette Slots...etc "
    PLZ, come & join with us, Thanks for your many tutorials!

    gclub online casino

  4. This comment has been removed by the author.

  5. All-Star Game Online Betting
    GclubWe are one of the leading online gambling sites. The launch of this season's gambling games can be an advantage. With online risk. Because the Internet is an important factor in the daily life. Therefore, the opening of online betting services to provide more convenient. They can also be carried anywhere. Online roulette bets on all mobile phones. Enjoy online gambling at any time with the kind of easy bet. Not only this, gambling online also gives you real money. Can be deposited - withdraw immediately. Challenge all the gambler's abilities. Whether you are playing gambling online, you have to have fun in the game. Because we meet all the needs of gamblers. Go through the page per page or to download the program. Online gambling does not have to risk being arrested. You will be able to enjoy the continuous festivity. If one of the gamblers looking for new gambling. That makes the player more than joy. Royal1688

  6. Entertainment is waiting for you on the online casino site.
    Ruby888 Online Casino Site Is a new game Will not make you monotonous. And to be boring anymore. You can also choose to play at any time, even if you have your own password. You can also accumulate points. The game of the casino online. There are a variety of games and ways and ways to play different. And there are recipes that will make you a great champion. But how? Your own playing expertise will determine whether you will win or lose each game. Online casinos also offer prizes and promotions, as well as good activities. Always give you Only you can join. In our online casino. You will be able to get that right now. And you will be happy with the game. You are the one who set the prize for yourself in an unknowingly boring. Do not decide whether the online casino is a scam or not. Because our online casinos actually pay real money, you get really rewarding. You do not have to be tired. And playing games is no longer a waste of time. If you are interested, please visit our website. IBCbet

  7. Wow, absolutely fantastic blog. I am very glad to have such useful information.


  8. Very nice post really ! I apperciate your blog Thanks for sharing,keep sharing more blogs.


  9. The most popular destination for Hadoop Training in

    in NOIDA is CIITNOIDA in Sector-2

    You can Master the various components of Hadoop ecosystem like Hadoop, MapReduce Architecture, Pig, Hive, HBase, Sqoop, Flume by their Senior Experts

    from Industry with good Industrial experience of handling big projects.

    You can even Get hands-on practice with live projects and ORACLE certification at CIITNOIDA.

    This Big Data Hadoop Training in Noida is best

    suited for CS & IT Engineering Students, professionals looking to gain expertise in Big Data and work for Fortune 500 Software Companies.

    Hadoop Training in Noida
    Big Data Hadoop Training in Noida

  10. The blog or and best that is extremely useful to keep I can share the ideas of the future as this is really what I was looking for, I am very comfortable and pleased to come here. Thank you very much.

    Digital Marketing Course in Chennai
    Digital Marketing Training in Chennai
    Online Digital Marketing Training
    SEO Training in Chennai
    Digital Marketing Course
    Digital Marketing Training
    Digital Marketing Courses

  11. I have need for more Knowledge for SAP .
    I`m Always read you Article .

  12. I’m bored at work, so I decided to browse your
    site on my iPhone during lunch break. I have read your post.It was useful & interesting.So keep it up. Chartered Accountant
    Stock Audit
    CFA Audit
    Warehouse Audit

  13. Extremely it’s an astonishing article I had ever read. I hope it will help significantly after all. Thanks for sharing. Keep it up. Duplicate Payment Review
    Continuous Transaction Monitoring
    AP Vendor Helpdesk

  14. The post was really good. Thanks for sharing
    shriram earth plots

  15. This comment has been removed by the author.