February 2021 marks the 20th anniversary of the signing of the Agile Manifesto. Follow along as we reflect and look to the future.
Take a moment to think about where you were in February of 2001—at the inception of the Agile Manifesto. Some were not born yet, others mid-flight in our IT careers, and still others moving toward our mature technology years.
I was with a privately held software engineering firm focused on helping organizations transform their software testing capabilities. We worked with Fortune 1000 organizations to review their current state from a people, process, and technology perspective, make recommendations for improvement, and help them amp up the effectiveness and efficiency of their testing. We called it “Test Transformations”.
This month, February 2021, we celebrate and appreciate the seventeen team members who came together in Utah and helped us “uncover better ways of developing software by doing it and helping others do it.” In reflecting on these past two decades from a software testing perspective, let’s look back, ponder how we’ve progressed, whether agile has helped software test engineering, ask ourselves to what extent we’ve advanced our craft, and consider the evolution of our testing capabilities.
Here are some highlights of our journey as a software testing profession since the creation of the Manifesto (compliments of Google and testingreferences.com):
2001–2010 Software Testing Highlights
Stickyminds.com launched, software attacks concept introduced, framework for integrated testing discussed, ISTQB founded, IBM Rational Functional Tester released, 1st generation of record/playback hits the streets, Selenium developed, soap opera testing discussed, STP magazine launched, SPICE (ISO 15504) published, Domain-Driven Testing idea, Jira 1.0 released, scenario testing introduced, Test Driven Development defined, Agile Testing Quadrant defined, Test Maturity Model (TMMi) published, and since their inception in 1992, STAR East and STAR West Conferences continue. Initial World Quality Report published, Test Automation Pyramid defined, Software Testing as a Service thought of, Software Testing Club founded, ISO 29119 working group formed, testing vs. checking debated, uTest launched, Weekend Testers founded, T.E.S.T magazine launched, modular, data-driven, library and keyword-driven frameworks introduced, Behavior-Driven Development discussed, several additional Testing Conferences launched; First CAST (Conference of the Association for Software Testing), Agile Testing Days, Belgium Testing Days, and we discuss and debate agile topics at STAR conferences.
2011–2020 and Beyond
Testing Is Dead declared, Specification by Example, Let’s Test Conference, This Week in Software Testing podcast, distributed software testing using cloud computing, IEEE/ISO/IEC 29119 released, Test Centers of Excellence (TCOE) embraced, crowdsourced testing, mobile testing, IoT testing, Quality-Driven Development, the acceleration of open source testing capabilities, shift from central test teams to embedding testing roles in agile teams, continuous testing, continuous integration and DevOps. AI/ML in testing, full stack test automation frameworks, continued focus on shift-left testing, testing in the cloud, increased focus on security, user experience, and accessibility testing, increased comfort with testing in production, continued acceleration of the need for testing roles to demonstrate software engineering skills (SDET’s), big data testing, blockchain tests, QAOps, more focus on autoscaling and parallel testing, chatbot testing, and leveraging Robotic Process Automation (RPA) and Hyper Automation (a blend of AI, RPA, and ML).
With these aforementioned memory joggers, a few questions come to mind:
- Has the introduction of agile helped or hindered software testing?
- Have we, as software testing professionals, advanced our craft?
- Is the art and science of software testing keeping up with the pace of change in technology?
My pessimistic answers to these three questions are: No, No, and No. My more optimistic responses are: Yes, Somewhat, and No.
Has the introduction of agile helped or hindered software testing?
On the downside, agile’s initial introduction continued to leave testing roles out of the conversation. In fact, some interpreted the manifesto values and principles in ways that negated planning, documentation, and testing. In fairness, as test professionals we have some accountability for being slow to embrace change and demonstrate leadership. Some implementations of agile continue to be waterfall with a few agile words thrown in. The emphasis on “just automate everything” or “testing is dead” fully undermines the importance of critical thinking, intelligent test design, a risk-based approach, exploratory testing, and the need for qualified software testing professionals that strive to master their craft. I would assert that agile has not significantly increased the perceived value of testing or the testing profession, nor has it reduced the need for comprehensive testing in our increasingly complex solutions.
On the positive side, the adoption of agile values and principles in organizations has helped shift testing left, drive toward whole team accountability for quality, contributed to earlier defect identification, helped us adapt and respond to change more quickly, lowered risks, shortened feedback cycles, and helped us learn faster. It has increased our focus on delivering customer value, removed the barriers between teams, increased collaboration between roles (particularly dev and test roles), amplified the definition of done, accelerated the adoption of test automation, helped us drive testing to lower levels of the system, and for many has positively contributed to the business outcomes of improved quality and customer satisfaction.
Have we, as software testing professionals, advanced our craft?
Heck no—or maybe somewhat. We do have multiple tools in our toolbox—from methodologies to processes, techniques, and practices, to open source and commercial testing technology that we have developed. TDD, BDD, and exploratory testing, test design approaches like equivalence, boundary, decision tables, state diagramming, risk analysis processes, risk-based testing, measures and metrics, entrance and exit criteria, definitions of done that include testing, quality gates, both manual and now automated in DevOps, a wide range of training for software testing roles, the ability to dynamically spin up test environments both on premise and in the cloud, and much more.
HOWEVER, the rate of software creation continues to significantly outpace our ability to verify and validate software capabilities. Simply put, we continue to produce more code each day than we can test. Innovations in software test engineering continue to move at a much slower pace than software engineering, which is closely related to my next point.
Is the art and science of software testing keeping up with the pace of change in technology?
No, no, and definitely not. Did I say NO? I believe our journey toward software testing excellence remains in its infancy. Opportunity abounds for “uncovering better ways of TESTING software by doing it and helping others do it.” The methods, processes, techniques, practices, and tools I previously mentioned have been around for a decade or more. Commercial and open source test tools struggle to keep pace with advancements in technology and the need to accelerate both the breadth and depth of testing. Outside of the application of AI/ML to testing, what innovations directly related to software testing come to mind over the last few years?
The Agile Manifesto values and principles have contributed significantly to accelerating the quality delivery of value to our end users. Our continual challenge is to ensure that as we transform our digital culture, agile is inclusive of all disciplines. Each discipline within our value streams must take accountability for driving revolutionary change, and our software testing profession is no exception. Let’s celebrate and appreciate two decades of improved agility, but let’s keep challenging ourselves to innovate and accelerate our technology and approaches to software test engineering. Our collective goal remains the same—deliver value to our users with excellence in a timely and cost effective manner.
As we begin our third decade of agile, it is imperative that our software testing profession accelerates the experimentation and innovation in our craft to help close the gap between code creation and code assurance.
