Writer and tech entrepreneur Derek Sievers famously said that ideas are a multiplier of
execution. An average idea paired with excellent execution will go further than a poorly executed brilliant idea.

A crucial part of the execution of any software product is ensuring it works as expected.
MVP-stage startups might be able to get away with buggy software, but as more and more
users begin to rely on a product, its stability becomes paramount.

Testing software before shipping it to customers is a vital part of the application development
life-cycle. In this post, we’ll look at different kinds of software testing strategies, their benefits,
and when to adopt them.

Manual Testing and Its Downsides

There are different types and flavors of testing, but the most straightforward is manual
testing. Launch your app, interact with its interface, and make sure it behaves correctly.

Unfortunately, manual testing doesn’t scale. As your product grows, it experiences a
combinatorial explosion in possible user journeys. Manually testing each scenario before
every new release can end up taking days.

When manually testing, we look at the software with our eyes and use our brain to verify its
behavior. Unfortunately, the biological hardware on which we run manual tests is not
particularly fast.

If speed wasn’t enough of an issue, humans are inherently flawed machines. We distract
easily, are afflicted by all sorts of biases, and our performance depends on unrelated factors
such as how long we slept the night before or our emotional state. We are bound to make
mistakes and miss a detail.

Luckily, there are testing techniques that shift the load from humans to computers.

Developers build products with code that automates tasks for our users. Nothing stops them
from writing code to automate the task of testing other code. We call these techniques
automated testing.

Automated tests are faster and cheaper to run. They are easily repeatable, and everyone
can run them. Automated tests scale much simpler than manual tests because they can run
in parallel on multiple machines.

There are different types of automated tests operating at various abstraction levels. Software
and test engineers combine them to produce extensive and reliable test suites that ensure
their applications always behave as expected.

Unit Testing

The most common form of automated testing is unit testing. These tests work on isolated
logical units, usually a single function or a method in an object.

A unit test has three common stages.

• First, it arranges an input for the system under test.


• Then, it acts on its unit, which produces an output from the given input.


• Finally, it asserts that the output matches the expected system behavior.

Unit tests are fast to run and give developers quick feedback on the effectiveness of their changes.

Some developers lean heavily on the fast feedback-loop that unit tests provide with a
practice called Test-Driven Development, TDD for short. In TDD, developers write tests
before the application code. These tests necessarily fail, and the way in which they do
provides a hint on how to start implementing the code that will make them pass.

Due to their sharp focus, though, unit tests cannot guarantee full coverage of the end-to-end
application behavior. End-to-end behavior coverage is where integration and UI testing come
into play.

Integration Testing

As the name suggests, integration tests verify how individual units work together to achieve
the desired outcome. Usually, these tests leverage the same infrastructure on which unit
tests run. That is, they have access to the application’s internals but verify the behavior of
multiple units composed together, with little interest in the role each component plays.

To understand the difference between unit and integration tests, consider a common feature
such as logging into the app. This functionality requires components such as a low-level
networking module, an API client, and a view abstraction object. We can write dedicated unit
tests for each component to verify the different facets of their behavior. These tests will
instantiate a component and work on it in isolation, agnostic of the other parts of the system.

An integration test for the login will instead put all the pieces together and verify higher-level behaviors. One such test might ensure that when the view abstraction receives the signal to
start the login with a username and password that the API client would report as incorrect, it
will eventually present an error alert to the user.

Integration tests operate at a higher level of abstraction than unit tests because they verify
how individual pieces collaborate. But there is an even higher vantage point from which to
run tests: that of the user.

UI Testing

UI tests interact with the software through its user interface, simulating the actions a flesh-and-blood user would perform, such as tapping and swiping.

These tests treat the software as a black box: they interact with it but don’t know its internals
or access to them. That’s not the case with unit and integration tests, which instead depend
on having access to the individual components and are therefore referred to as white-box
testing approaches.

UI tests are an excellent tool to verify the core user journeys from start to finish. On the other
hand, because they simulate organic user interactions, these tests are slower and
sometimes unstable, or flaky, due to factors such as networking delays or UI animations.

Together, unit, integration, and UI tests can provide thorough automated test coverage for
most of the application’s behavior. After establishing a latticework of unit, integration, and UI
tests, developers can tactically deploy other testing techniques to further ensure the
software’s stability.

Other Testing Techniques

Snapshot tests verify an application’s UI details by taking screenshots and comparing them
against a pre-recorded baseline. They help prevent interface regressions and are also useful
to ensure the layout renders as intended in edge cases such as with very long strings.

Performance tests run subsets of the application, from individual methods to complete
flows, multiple times and compares the average execution time with a predefined baseline.
Developers reach for performance testing to ensure computation-intensive parts of the
applications don’t take too long to run, which would hamper the user experience.

Pact tests are tests specifically targeted at the boundary between an application and its
remote APIs. Because APIs and application usually live on separated codebases they may
get out of sync. For example, the API might respond with a list where the application expects
a single element. Pact testing detects and prevents this type of issue.

Client applications and API providers make a pact on the data format for each request and
response, described through a shared code interface. Developers then write dedicated tests
to ensure the contract is respected.

The advantage of pact testing is that it doesn’t require interaction with the other party,
meaning it bypasses external factors such as network latency and availability. Thanks to the
upfront investment in defining the contract between API and consumers, developers can run
more reliable, isolated tests at a much faster pace.

Manual QA and Exploratory Testing

The sad reality for many people working in quality assurance is that they too often operate

as manual testing machines. They receive checklists of scenarios to verify and painstakingly
run through them, a terrible underutilization of their talent and skills.

When the development team has invested in automated testing, QAs no longer need to
churn through manual testing checklists. They are instead empowered to focus on
exploratory testing.

In exploratory testing, a QA engineer proactively searches for ways to break the application.
Unlike all the testing techniques we’ve seen so far, exploratory testing discovers but doesn’t
verify issues. To hunt for hidden misbehaviors is a much more challenging task than merely
replaying a given sequence of steps looking for an expected result. More importantly, it’s one
that only creative humans can perform.

Exploratory testing is vastly valuable as it can prevent nasty issues from reaching users.
Issues discovered in this way are often deep-rooted. Fixing them improves the quality
of the software architecture itself, making it more stable for the future.

Where to Start with Automated Testing

With many different flavors of testing, it can be daunting to know where to start.

Whenever implementing new code, I recommend adding unit tests for it. There is no better
time to write tests for a component than while building it.

Software developers spend most of their time modifying existing code, though, not writing
brand new features. When working on an established project that doesn’t have good test
coverage, it’s best to add UI tests to cover an area of the app before changing it. I suggest
starting with UI tests because software lacking good test coverage often has components
that are hard to test in isolation. With the UI tests in place, developers can be confident the
desired behavior will be respected and start rewriting individual units to make them easier to test.

Adding tests to code that doesn’t have them before changing it establishes a virtuous cycle. Each test you add makes adding further tests easier. Over time, you’ll build an extensive test
network.

Finally, remember that the value of tests is in the feedback they give. When in doubt, start
with whichever test is easier to implement. Having that in place will give you the confidence
to further iterate on your design.

As Spotify CEO Daniel Ek wrote in a shareholders report, “speed of iteration will trump
quality of iteration.” A thorough and varied automated test network will empower your team to
iterate quickly without compromising your product’s stability.

---

Gio Lodi is a writer of software and words. He lives in an Australian beach town with his wife
and two children and works remotely as a mobile infrastructure engineer for Automattic. He’s
the author of Test-Driven Development in Swift. You can find him on Twitter at @mokagio and on his testing, automation, and productivity blog mokacoding.com.