Despite its incredible importance to ensuring a quality product, testing has traditionally been relegated to one restrictive phase in the software development lifecycle.
Thankfully, that thinking is changing.
As more and more organizations evolve their agile, DevOps, and continuous development strategies, software development leaders are taking a hard look at the role testing plays in these processes–and where organizations might be failing.
One person leading this charge is Janet Gregory. For nearly 20 years, Janet–along with her collaborator Lisa Crispin–has been pioneering concepts in agile development and testing through industry-leading books, training courses, and community engagement.
In a recent conversation, Janet talked with Coveros CEO Jeffery Payne about her concept of holistic testing and its role in ensuring continuous, quality product development. (Watch the full conversation here.)
Here are four key takeaways from the conversation.
1. Testing is not a phase of the development lifecycle. It’s an activity that should be ingrained in every part of the process.
The concept of holistic testing builds off the idea of continuous testing within a DevOps loop but rethinks the stages of the development lifecycle.
One of the key pieces of Janet’s concept of holistic testing and her ideal development loop, is that testing isn’t a phase in that loop. Instead, testing should be ingrained in every phase.
Janet quotes agile leader Elizabeth Hendrickson to sum up this concept: “Testing is not a phase. It’s an activity that happens throughout.”
2. Holistic testing aims to encourage a comprehensive strategy for how and what we test.
The holistic testing model isn’t restricted to testing of applications specifically. As CEO Coveros Jeff Payne points out, “You’re talking about testing not only the application, but any code or environment or system that your application must be successful on.”
This is one reason why Janet says she’s actually stopped using the term “software testing,” using holistic testing instead–to ensure that testers and developers don’t get caught in the “bubble” where they’re focused only on testing the application being built and not other key pieces of the development process.
While holistic testing aims to expand our thinking about what needs to be tested, it also aims to be comprehensive about what types of testing is necessary.
For Janet, the concept of continuous testing has become too synonymous with test automation. Janet’s goal for holistic testing is to re-introduce the vital, human-centric tests that need to be run alongside or apart from automation.
3. When we think about the development process, we should remap our mindset to “test AND code.”
One of the big pitfalls of traditional development approaches, according to Janet, is the idea of throwing code over the wall for someone else to test. Instead, she argues we need to be thinking about coding and testing as one, intertwined process.
Janet also argues that the concept of “code then test” leads people to assume that testing comes after coding is already complete. If we approach testing and coding as an intertwined process, however, we should be thinking about testing at the same time or even before coding.
Bringing these processes in closer alignment will encourage better communication and collaboration between testers and coders and more efficiency to the development process, Janet says.
4. Test automation is a powerful and helpful tool, but it should be used strategically.
“When we think of automation as a tool,” Janet says, “I want to get the best out of it.” She says organizations should be focused on automating tests for issues they know exist, know how to test, and need to test repeatedly.
The closer we test to the code, Janet says, the quicker the feedback is that we receive. Good areas to focus test automation might be around things like business rules at the API level.
When it comes to implementing test automation, understanding what it takes to maintain that automation is key, Janet says.
Jeff Payne relates this focus on repeated testing to the concept of toil in DevOps or Lean. If we can focus automation on things that are done manually, over and over again, and that would be better suited for automation by machines, that gives humans more time to think more strategically and critically about what else needs to be tested.
For more on Janet’s concept of holistic testing, be sure to watch the full conversation here.
