The fundamentals of testing ...

I've been working to try and capture the fundamental aspects of testing, and what our our "prime directives" and place within projects should be.

Here's a revised version of what I feel are the essential drive and vision for a testing department.  Whatever your product, whoever your customer is, you should find yourself nodding along to the majority of this.


The value of testing

Testing is important. It holds the key to safeguarding the quality and reputation of services from your organisation.

Projects most testers will encounter during their career will be diverse in their ambition, scale and partnerships. Thus a one size-fits-all approach will not work, but there are some essential traits which testers should identify as necessary for their role.

Mission statement


Testers should be dedicated to serving and championing its customers and business users. To that end the greatest customer experience is quality and trust from our customers in the services we provide.

Testing in your organisation should be dedicated to ensuring that any product you deliver will, wherever possible, continue to support your customers trust in your organisation.

Your aim is to understand, document and assess any risks within a new system, and aim to prevent any surprises when that system goes into production.


The Principles of Testing


The following principles should be upheld and championed by testers,

Objectives

• The aim of testing is to ensure products are fit-for-purpose and to enable them to be delivered.

Assisting Project Management
Testers should,

• Use their experience within the software industry to mentor Project Management on the testing process, as well as seek to support the decisions of management.
• Regularly report to project management regarding progress of testing activities and expected timelines.

Defects
Testers should,

• Work to illuminate high impact defects rather than report cosmetic defects en mass.
• Work to verbally communicate directly to development whenever this is possible.
• Communicate any high impact defect project management ASAP after it is raised. A similar attitude should be taken to any holdup which is delaying testing activity, such as delays receiving documentation or test environment not being available.
• Work with development and software providers, not against them. Testing is be an exercise done in partnership with these people to ensure the highest quality of product possible.
• Clearly communicate problems within defects.  Defects should include any reproducible steps, together with reference to requirements/design where possible.

Testing activities

• Early access to any software under development will help both scripting and early feedback on software.
• To best understand a product and it’s intent, software testing will need access to business analysts, technical architects and developers as well as any relevant requirement and design documents which allow understanding of the planned product.
• Where behaviour of a planned product is complex, testers should take leadership and ensure design walkthroughs occur.  Here the functional behaviour will be talked through with business analysts, architects, developers to ensure a working knowledge of the proposed product.
• The scope of testing should be scaled according to it’s appropriateness for this project rather than because it was done on another project.
• Test scripting should add value, and function as a repository for future regression testing.
• Wherever possible, testers should attempt to verbalise any issues encountered before formally communicating them.

Doing things better
Things don’t always go well on a project. Testers should be committed to learning from past mistakes to avoid repeating them. But neither should they overreact.





Deliverables
The following deliverables should be provided by testers for any test project,

• Test strategy document – initial scope and estimates for each testing phase applicable to project defined early on in the project, from which test budget can be calculated.
• Detailed test plan – based on test strategy, this is a more comprehensive plan for a particular phase based on available business/technical requirements
• Test scripts – a structured detail of what is planned to be tested / was tested on a software product
• Proof-of-testing – details of what was tested when and by who. Might be test script with tester name and date on.
• Defect report/notification – communication of any defect raised
• Test Exit Report – final document detailing what has been tested, what problems were encountered, what was fixed, what defects are outstanding, and why it’s considered acceptable.

Standard Risks
The following are standard risks which can affect testing timelines and budget,

• Late delivery of software for testing – the software for testing does not become available until later than planned, and hence testing is ready to begin, but cannot commence.
• Dependency on prior testing – within any test plan there tends to be dependencies that some testing has occurred beforehand. Otherwise testing will be the first point at which many defects are detected, impacting with a higher than expected level of retesting. Can be mitigated a little by asking for release notes of fixed defects, and test exit reports from prior phases which illustrate which issues were encountered and resolved.
• Time erosion – has any events which take testers off-task such as meetings been incorporated into the test plan. Meetings are a vital forum for understanding and communicating issues on the project, but the level of meetings needs to be known, and made sure to be “just enough”.
• Requirements – is there a single source of truth for technical, business and design requirements to cross reference? Are the people who wrote these documents available? This will help reduce unnecessary defects being raised.
• Build turnaround – the amount of time it takes development to turn around a new build and address defects is vital. Most software, even essentially good software, requires 2-3 builds to get polished for release. If your developers can only agree to a build weekly, and you have 3 days retest planned, this will not work. There need to be planned software release drops.
• Test environment – has the designed architecture been suitably understood that the correct test environment has been ordered and set up?
• Mis-estimation of scope – has an element or system not been assumed to be part of the test effort, but actually is? This will cause more to need testing than expected. This is why a Detailed Test Plan needs to revise initial estimates, and re-review in line with signed off Business and Technical requirements when they become available.