With the rise of DevOps movement, there is an increased focus on delivering automated, continuous improvement.

Continuous Integration (CI) and Continuous Delivery (CD) are core software development disciplines that are part of an Agile software development cycle. The idea is to release fully-tested and stable builds early and as often as possible. Automation, specifically automated testing, is a crucial component of this process. In short, Continuous Testing is the core enabler for CI/CD.

Introducing testing early on the “left-side” of the development process, starting with unit tests and complemented with service/functional and integration tests, is fundamental for making CI/CD happen. But that may not be enough. While Shift-Left Testing happens during the development, Shift-Right Testing happens in post-development scenarios (i.e., in or based on production data).

Let’s examine the differences between the two.

Shift-Left Testing

To facilitate CI/CD, testing needs to be prominent from the start, specifically at the requirements specification/story definition phase. Shift-Left Testing is crucial in Agile teams as it allows artifacts to be properly validated through clearer and better communication. The idea is to find bugs and fix them as soon as possible. The sooner, the better.

Activities:

  • Discussion of requirements (either the typical requirements or the Agile’s user stories), bugs and any other artifact that implements, changes or fixes behavior of the product
  • Writing of unit tests at the same time the artifacts (i.e., issues) are being implemented by developers
  • Writing service/functional/integration tests (by the developers or the members having the “tester” role); this can be done right after the issue (e.g., story) is clarified and included in the Sprint backlog
  • Test execution, which happens as soon as possible (during the Sprint in case of Scrum); this execution is hopefully automated and unit tests run often, upon code commits or merges. Other types of tests (non-UI or end-to-end) run also as soon as they’re implemented.
Pros Cons
Better and clearer requirements/stories Lack of UI/end-to-end validation
Leads to the development of testable features => increase Testability Misses “real feedback” (i.e., coming from real users on real usage scenarios)
Bugs detected sooner
Reduce fix costs (saves time and money)
Prevent bugs at the end of development cycle/release moment
Introduces and embeds quality from the start
Reduces gaps between developing and testing
Early code review of automated tests, letting testers review assertions and usage scenarios

Shift-Right Testing

The focus of Shift-Right Testing is to get feedback from real users after development is done. It’s like extending and continue testing after the release has occurred.

To achieve this, techniques such as A/B testing, Canary releases or Blue/Green deployments are used to obtain feedback from end-users about the delivered features in a controlled manner.

Active monitoring tools are used in order to detect errors as soon as they appear and also to gather valuable performance metrics.

Because development is done, and all product features have been integrated, end-to-end testing, including UI testing, can be done properly.

Pros Cons
Use direct feedback from users More costly bug fixes
Allows automated testing at the UI level with end-to-end workflow testing
Features are complete, more integrated and more stable
Allows real performance, stability and security testing
Discover obscure usage scenarios
Test ideias and UI changes with A/B testing
Detect bugs in controlled scenarios
Enables Blue/Green deployments and release rehearsals

Which testing strategy should you follow?

Agile teams would do best with Shift-Left Testing for clearer specifications, smoother implementations, earlier testing, and reduced costs. However, Shift-Right Testing adds other benefits, such as the ability to incorporate user and system feedback based on real usage scenarios.

Therefore, we see these approaches not as exclusive or contradictory, but complementary.

As depicted in the Test Pyramid, you can focus your effort more at the “ground level” with unit testing, but that does not imply that you shouldn’t do other kinds of tests at different levels. The same is true for Shift-Left/Right Testing: you may invest more in requirement reviews and unit tests, but that does not imply or exempt you from doing integrated tests afterwards, especially if they involve the actual end-users in actual usage scenarios.

FacebookTwitterLinkedIn

1 comment

Leave a reply