There isn't really "one type" of automation to assist testing. However when we talk "test automation" most people typically mean "GUI automation".
It's important to learn about the different types, which we should cover by the end of this series. Using the right automation task for the right job is important - as someone once told me "technically you can use a drill as a hammer ... but it's not a very good hammer".
Today we're just going to focus on a high level description and definition for each. We'll spend quality time with each before moving on in later pieces.
Unit checking
Our friend Wikipedia says: unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested
That's kind of true, but also a little bit confusing. I actually don't like confusing, because I will admit, I'm easily confused.
These days, most modern languages allow you to build in some forms of test into your code as you build it. These tests allow you to check your code at it's most basic level. It also allows you to run these checks incredibly fast. There's an example coming up which will really help to illustrate this.
A great example of languages which allow "unit tests" is Java - look up the JUnit test framework, whose checks run as part of the build process (so developers find out really fast if there are problems).
API checking
Wikipedia says: API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security.
Most solutions today are built in layers or components. They are a mixture of commercial off the shelf components, and bespoke development. These components typically communicate between themselves using APIs, which are a form of information protocol.
Automation with an API tool allows you to perform checks on a system, by injecting control to a component directly using the API protocols. Almost as a kind of remote control. You can also get output via the API to check against.
It also allows you obviously to actually perform some activities in a system where a vital component is missing. This component could be missing because it's not ready (those rascally developers doing the GUI, when will they learn?) or because it's something you can't simulate in your test environment.
A good example of a tool for API testing is soapUI - which out of the box is actually fairly manual, but mix it with some coding, and you've a powerful platform for API tests
GUI checking
This is the kind of automated check most often thought about when people think automation.
Wikipedia says ... well quite a lot really, and not easy to summarise. Have a read yourself if you want.
I'll defined a GUI check as one which operates, manipulates and evaluates your web system using the actual front-end of your website. I would like to add "as if an invisible entity was controlling", but I don't want to get burned for witchcraft. Although ...
It has to be said, there's something odd when your tool allows you to watch the testing in action - it does look like you're machine is possessed.
Compared to unit or API tests, GUI tests are much slower (not many orders of magnitude faster than when I manually test), and have a tendency to break and need a lot of maintenance. Small changes to the page design (or worse still, a bad design), will have you constantly tweaking your suite of scripts.
One example of this is of course Selenium IDE which is a record and playback tool. This means you "hit record" and it learns to repeat back your actions during recording in a very basic XML script.
Such tools can be a danger unless you know what you're doing, because they don't make use of real strengths of a scripted language. A good example is every time you create an account, you'll use slightly different data as a manual tester, these record and playback tools struggle with this.
Selenium Webdriver is much more powerful tool as it can be controlled by a programming language, taking advantage of many of the flexibility of such languages. We'll explore later what that opens up for us.
Performance Checking
Before we go it's useful to just cover off performance tools. Of all the tools, these are the ones I've the least hand-on experience with, but I have managed such testing, and do understand the principles.
In a nutshell they allow you to simulate webtraffic - JMeter for instance will send html requests to your system which simulate certain functions such as "registration", "login" or "transaction", and measures the response time from the system.
The faster / more requests it sends, typically the slower the response from the system. Typically you're looking to see if it breaks down and become unacceptably long at certain loads.
Check out Oliver Erlewein's articles on Hello Test World for JMeter here.
This comment has been removed by the author.
ReplyDeleteThanks Jim - well I learn something new every day.
DeleteI chose zoology as I feel (certainly in what's to follow) we'll be studying them as different beasts. But yeah, this is probably closer to a taxonomy.
For API testing, I like Postman, or just using cURL. Isn't SoapUI more for soap services testing (I know almost nothing about soap)?
ReplyDeleteI find the test automation pyramid (originated by Mike Cohn, here's one of his posts: http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid) helpful when thinking about these kinds of automated checks.
I like taxonomies like this. Also models like the pyramid and the agile testing quadrants. Take this list to your whole delivery team and talk about what automated regression checks that you need for your product.
I'm doing a broad view here, and we're going to look a little closer at each. The pyramid is something I want to cover, it ties in with one of my iron rules. But I want to do so once we've explored these test types a bit more - ie SPOILERS! ;-)
DeleteDon't take the Automation Pyramid as gospel. The way it is shaped comes from natural process and not some pre-planned methodology. Of course the base will be wider because of the nature and type of automated test (checks or whatever, splitting hairs in my opinion and remember what I said about splitters and clumpers) they are. Unit Tests are by nature singular in purpose and focus. Because of that you have a one-to-one relationship with the code itself and thus you will naturally produce more of them (thus the wider base). They are more atomic. Finally, these types of tests validate single conditions. Benefit of all of this is that a developer is testing their own code for correctness and robustness. It helps to eliminate "stupid" bugs in code sooner and alleviate the headaches testers later in the process incur when sloppy code is delivered. The next layer up is an integration layers where it is testing whole pieces of functionality and interactions with them. Not as atomic, but important because you need to know function and other programming interfaces are sound. This can be functions within the app, or functions between apps. This helps with proving robustness and reliability of the code. Finally at the top are Acceptance Tests (which can be UI or API/Services layer too). These are more process driven tests and if the other layers are in place you don't have to build as many as the other layers to test the software overall. You focus on proving things at the macro level.
DeleteSo if you are following a test driven development approach you will naturally build out your automation approach like the pyramid. I tend to think of it all as concentric circles of coverage, you build upon the other to provide the coverage of testing that you need.
Jim