UATP Revision
I reviewed our UATP process, and suggested changes which reduced our scoping time by an estimated 18%.
Rethinking UATP: 18% Reduction in Scoping Time
At a Glance
Context: Our UATP process was time-consuming and ineffective
My Role: Team Lead, UX Designer
Results: I researched our process and overhauled our approach, reducing project scoping time by 18%
Below: At Start Studio, we replaced our spreadsheet-based UATP with an approach using annotated design specs.
Intro
I remember a friend showing me a poster he’d created using Excel. I was at once amazed and appalled — amazed at the creativity required to get this result from that tool, and appalled that he hadn’t availed himself of any of the many tools better suited to that purpose.
Such was Start Studio’s first attempt at User Acceptance Test Plans (UATPs). Our approach postponed development, increased our clients’ costs, and went ignored by the people it was meant to assist. I reviewed our UATP process, and proposed solutions, which reduced scoping time by approximately 18%.
The problem
While working on Speiro, a construction management app, I was asked to update the UATP for the project. I knew we had started doing UATPs, but I didn’t know anything about them. As I scrolled through the 215-line spreadsheet, my eyes glazed over, my heart rate slowed to hibernation levels, and I suspected that nobody was using this document.
Imagine reading 215 lines of this.
The Approach
Start Studio’s scoping phase included our entire design process, as well as writing a UATP, and estimating development costs. I reviewed Speiro’s scoping phase, and found that writing the UATP accounted for 18% of the total time, second only to wireframing. I questioned the efficacy of the time spent creating this spreadsheet.
For the Spiero project, writing the UATP accounted for 18% of the scoping phase.
I interviewed Dave, who was tasked with writing our UATPs. I also talked to the developers who were expected to benefit from the test plan. I found that no one was actually using the UATP. One developer called it “a deterrent to go through,” and even Dave acknowledged that it was “extremely boring to write.”
We were spending significant time and money to create these spreadsheets that went almost completely ignored. There was agreement that we needed a source of truth for how things should work, but our current approach was ineffective.
I summarized my notes, and identified areas of consensus. The primary problems included:
Lack of usability: The spreadsheet was painful to read, and developers saw it as an impediment to their work.
Too broad a scope: The UATP was trying to accomplish everything from providing a basis for estimating development, to being a legal document the client would sign off on, to verifying project completion.
Redundancy: The UATP didn’t complement the designs, in some cases trying to verbally describe the visual elements in the designs.
Imprecision: The UATP could be vague, recommending that components “work properly” without defining what working properly looked like.
Opportunity Costs: Our one QA person was spending much of his time writing UATPs. Time that could have been spent testing and improving our products.
I realized that many of the problems with the UATP could be addressed via notes to accompany the designs. I had already started adding explanatory notes to my designs, which reduced the back and forth question-and-answer time between design and development. I recommended that, in most cases, additions could be made to the design notes to accomplish the purposes we were trying to address with the UATP. This would have the following benefits:
Create a single source of truth for developers to reference. The combination of verbal and visual elements in one document allowed devs to easily understand intended functionality.
Provide developers with a more effective basis for creating estimates, because it included visual, as well as written, elements.
Provide clients with something to sign off on — much easier to understand and digest than a never-ending spreadsheet.
Provide QA with the basis for writing tests.
Free up QA up to write and perform manual and automated tests, improving the quality of our products.
Adding notes to the designs provided a single source of truth for development, estimating, and writing tests.
The Results
As a result of my efforts, we abandoned the spreadsheet approach to UATPs, in favor of annotated designs. Based on the time spent to write the UATP for Speiro, I estimate that this reduced the time of our scoping phase by 18%, passing those savings on to our clients, and moving projects to development faster.