Testing in an Agile Development Environment: TDD vs. BDD

Introduction

In a traditional environment development and testing are separated into two phases: developers who build features and quality assurance engineers responsible for testing them. Quality engineers write and execute detailed test plans for new features but might also find bugs in existing features caused by newly written code (regression testing).

An Agile development strives to constantly ship new quality features. But how can good quality at a faster speed be achieved with each build? Traditional testing methodologies simply do not work in that case. In order to keep up with that different rhythm of development, a new approach in ensuring software quality is needed.

Principles of Agile testing

Short feedback loop & automated testing

New or modified code must be tested before delivery. In iterative methods such as Scrum, this means that an Agile tester must provide the team with fast feedback – usually within two weeks – about the created or changed functionality.

In order for this to work out, functions must be available in a first testable version after just a few days. In addition, the tester is also involved in defining the function before the sprint.

To speed up feedback, many teams rely on a high degree of test automation in combination with lightweight manual exploratory tests. While programmers support the work of testers in automation, testers with their critical capabilities provide the basis for test automation and knowledge on edge cases.

Agile teams are aware, though, that not everything can be automated; and they address risks such as UX issues (poorly designed user interfaces), security, portability, robustness or any other aspect of quality that cannot be defined in a checklist by performing exploratory tests.

Collaborative testing

On traditional projects, the testers are responsible for all test activities.

In an Agile setup, the testing responsibility is shared among the entire team.

Especially the divide between software developers and software testers is one of the major issues that Agile tries to solve. A process where one party (developers) aims to write the perfect code prior to handing it over to another party (testers) who then seeks to break it in as many ways as possible is seen as largely inefficient.

Instead, developers are encouraged to think like testers, continually checking their code for potential errors, while testers are encouraged to think more like developers by taking part in the process of creating software itself. This requires, though, that developers enhance their testing skills and embrace direct interaction (pairing).

Testing is not a phase

Agile development considers testing – just like coding –  as an inherent element of software development. While the team is incrementally building each feature until it provides enough value to be released, testing and coding are done incrementally and in collaboration between testers and developers. A piece of code is written and checked that it works correctly before the next piece of software is built. Continuous testing is the only way to ensure continuous progress.

Reducing overhead & lightweight documentation

The effort for non-immediate operational test activities (test management, bug management, documentation tasks) should be reduced as much as possible. So Agile testers instead of writing lengthy test documentation tend to:

❏     Use reusable checklists to suggest tests

❏     Focus on the essence of the test rather than the incidental details

❏     Use lightweight documentation styles/tools

❏     Capture testing ideas in chapters for exploratory testing

❏     Leverage documents for multiple purposes

What is Test-driven Development (TDD)?

Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: Requirements are turned into very specific test cases, then the software is improved to pass only the new tests.

TDD tightly couples coding, testing (unit tests), and design (code refactoring), as follows:

1. Write a unit test describing a single aspect of the program
2. Run the test, which should fail as feature is still lacking
3. Write just enough code to pass the test
4. Rewrite the code if necessary

“Rinse and repeat” to accumulate tests over time.

Expected Benefits

Test-driven development has a range of advantages, major ones of which are:

❏     Faster feedback: TDD provides almost immediate feedback on the components an Agile team is developing and testing. Developers usually get closer and quicker to the root cause, spending less time debugging.

❏     Higher acceptance: Higher probability of matching the product owners’ vision for the user story, because test cases are directly derived from acceptance criteria (and focused on by developers first, rather than their focusing first on architecture design or programming constructs).

❏     Avoiding scope creep: Because test cases or unit test drivers define the exact set of required features, it becomes easy to detect redundant code and unnecessary engineering tasks.

❏     Modular code: TDD can lead to more modularized, flexible, and extensible code. The technique requires developers to think of the software in terms of small units that can be written and tested independently and integrated together later.

Those benefits do not come without challenges, though. Many developers find adoption of TDD practices difficult. Because one part of the overall development effort is moved to the beginning of a programming phase, engineers tend to have the impression that their overall development speed has dropped, but generally they are able to pick up speed and save time when debugging in the end. Finally, the maintenance of the test suite itself can require additional effort.

What is Behavior-driven Development (BDD)?

In software engineering, behavior-driven development (BDD) is a software development process that emerged from test-driven development (TDD).

The concept behind BDD is to provide development and management teams with a shared process and shared tools, so that they can effectively collaborate while developing software. In order to accomplish this, BDD combines the basic principles of TDD with object-oriented analysis and domain-driven design to make the process of creating software as optimized and effective as possible.

It is an approach for building a shared understanding of what software to build by working through examples.

So how is this achieved? Behavioral test cases are written in a non-technical (ubiquitous) language that everyone can understand. Requirements for the software are described using examples, so-called ‘scenarios’. Usually, a specific format is defined for the description of these scenarios so that the automated review of the scenarios can be easily implemented later.

A commonly used format is the description language Gherkin, which follows a certain pattern containing the three elements context (the starting state), event (the users’ actions), and outcome (the expected results):

• ‘Given’ describes the initial context for the example
• ‘When’ describes the action that the actor in the system or the stakeholder performs
• ‘Then’ describes the expected outcome of that action

Take the example for getting cash at an ATM:

Given the account balance is $100

And the card is valid

And the machine contains enough money

When account holder inserts card and requests $20

Then the machine should dispense $20

And the account balance should be $80

And the card should be returned

The above example uses “ands” to combine multiple expressions within each clause, which is very common for business-facing tests.

The biggest advantage of behavioral tests might be that they describe a set of functions that a user expects from a system in a very concrete and direct manner. The sum of these behaviors essentially document a contract with the user/client. If any of the tests fail, this contract is not upheld.

While it can be quite difficult to write unit tests for existing code, behavioral tests can be written for a system at any time: before, during, or after development. The case where the tests are written first is termed behavior-driven development.

Benefits of BDD include:

❏     Better commitment and buy-in: By putting importance on business value, BDD encourages business owners to focus on concrete value and to prioritize accordingly. It also helps developers to accept priorities set by business because they understand the underlying thinking process.

❏     Shared understanding: Collaboratively writing scenarios in ubiquitous domain language, business and development teams gain a common understanding of what is to be achieved.

❏     Improved focus: Instead of jumping ahead and getting caught up in implementation details upfront, BDD encourages the team to focus on user needs and the expected behavior.

❏     Evolutionary design: Agile teams understand that not all requirements can be known upfront. BDD supports that iterative approach to product development, evolving system design and adapting to new user needs.

How do TDD and BDD compare?

In short, unit tests – a core element of TDD – test a single component in isolation, while behavioral tests – as elements of BDD – test an entire system. Some major differences are:

• Testing of a component in isolation requires mocking of external dependencies. While this requires some extra work when writing such test, running it is relatively quick. With a behavioral test, on the other hand, the system first needs to be put into a certain state before starting the test. Not particularly difficult but not fast either.
• Finding what caused a bug tends to be straightforward with unit tests while detecting the root cause when a system test fails can be time consuming.
• With code and test being tightly coupled, unit tests they are not portable whereas using a different programming language or changing the systems’ architecture does not necessarily result in a need to change a behavioral test.

An excellent, more detailed side-by-side comparison of both concepts can be found in Gray Straughan’s Cheat Sheet “TDD vs BDD”

Conclusion

So what to use when? In an environment that fosters team collaboration, flexibility, simplicity and constant feedback throughout the product development process, TDD and BDD complement each other perfectly.

With its ability to precisely detect failures, TDD focusses on code quality while BDD with its collaborative approach assures that the product corresponds to what is expected by the customer.

Meeting customer needs and delivering a flawless user experience by running tests with the right tools and using the appropriate methodologies: two important building blocks for companies to stay competitive in today’s fast-moving world.

References

On Principles of Agile Testing

Agiles Testen. (German) Wikipedia. https://de.wikipedia.org/wiki/Agiles_Testen

Engineering higher quality through Agile testing practices. (German) Atlassian. https://de.atlassian.com/Agile/testing

Engineering higher quality through Agile testing practices. Penny Wyatt. Atlassian.
https://www.atlassian.com/Agile/testing

On TDD

Test-driven Development. Wikipedia. https://en.wikipedia.org/wiki/Test-driven_development

Agile Testing Overview Redux. Testing Obsessed. Elisabeth Hendrickson. http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf

TDD. Glossary. Agile Alliance. https://www.Agilealliance.org/glossary/tdd/

Test-driven Development. Wikipedia. https://en.wikipedia.org/wiki/Test-driven_development 

TDD Research findings. Matt Hadley.https://weblogs.asp.net/mhawley/114005

 The Five Benefits of Test-driven Development. Brian M. Rabison. https://www.linkedin.com/pulse/top-five-benefits-test-driven-development-tdd-rabon-cst-pmp

TDD Cycle of Life. Agile in a Nutshell.http://www.Agilenutshell.com/test_driven_development

Advantages of TDD. Mark Levison. https://agilepainrelief.com/notesfromatooluser/2008/10/advantages-of-tdd.html#.WNP-2FPrs3w

The Five Benefits of Test-driven Development. Brian M. Rabison. https://www.linkedin.com/pulse/top-five-benefits-test-driven-development-tdd-rabon-cst-pmp

On BDD

Behavior-driven Development. (German) Wikipedia. https://de.wikipedia.org/wiki/Behavior_Driven_Development

Introducing BDD. Dan North. https://dannorth.net/introducing-bdd/

Given When Then. Martin Fowler. https://martinfowler.com/bliki/GivenWhenThen.html

Business Facing Test. Martin Fowler. https://martinfowler.com/bliki/BusinessFacingTest.html

Benefits of Behavior-driven Development. Naresh Jain, http://blogs.Agilefaqs.com/2012/11/11/benefits-of-behavior-driven-development-bdd/

The Beginner’s Guide to BDD. Konstantin Kudryashov. https://inviqa.com/blog/bdd-guide

3 Misconceptions about BDD. Juraci Vieira/Nicholas Pufal. ThoughtWorks. https://www.thoughtworks.com/insights/blog/3-misconceptions-about-bdd

On Differences between TTD – BDD

TDD vs. BDD. Development that pays. Gary Straughan  https://www.youtube.com/channel/UCVHTu50vmvVA0OtkGBJlGPw