Agile and user experience have a tortured relationship, especially with regards to usability testing. Some combine it with bug testing, while others skip it altogether: “We don’t have time for that.” It seems ironic to emphasize customer collaboration while cutting corners on customer testing to save a few bucks.
Agile is more of a philosophy than a playbook. The manifesto provides the basis for many different methodologies that all share the same “why” - e.g., ability to respond to change, more interaction between teams, etc. - but don’t specify “how.” There’s a lot of variety, and it’s reflected in how different teams handle UX.
How usability testing makes its way in can depend on the product owner’s approach or the organization’s UX maturity. When it works well, there is a common pragmatism that helps makes the tortured relationship more like a healthy marriage. The differences between Agile and UX are not irreconcilable; as with many things, attitude seems to be the key in making it work.
There are plenty of good guides out there on how to get user testing done in an Agile environment. Rather than list them all here, I’ll just offer a few Agile-friendly ideas to keep in mind:
1. Focus on “why”
Agile is sometimes called a faster way to develop software. This is not always the case, but speed is not the point. Smaller, more frequent deliveries are just one way to serve the goal of delivering better software. Usability testing serves the same goal, but the challenge is trying to get that testing done late in the game when designs are all but frozen.
It’s been my experience that how you talk about usability testing makes a difference in how or if it happens. At crunch time when everyone is scrambling to make sure nothing is broken, the team not have the stomach for talk of research protocols. User testing in Agile needs to focus on the customer and the business, so pare down the UX jargon and instead focus on what we intend to learn.
Consider an e-commerce website redesign. Behind every enhancement is a cause-and-effect hypothesis that a specific decision will result outcome:
- Will button label changes impact users’ ability to complete a purchase?
- Do redesigned search results help users discover related products?
- Can a new shipping page layout make it easier for users to upgrade shipping?
Focusing your testing on answering questions is a good way to highlight assumptions:
| Decision (“How”) | Assumption (“Why”) |
| Button label changes… | …will help users complete purchases |
| Redesigned search results… | …do help users discover related products |
| A new shipping page layout… | …can make it easier for users to upgrade shipping |
The larger the project, the larger those assumptions – and their potential to impact the business. Unless you validate ideas, the only way to know if a decision is the wrong one is through secondary indicators, such as phone calls and support tickets - at which point it may already be costing you money. Usability testing helps keep teams honest about how projects will affect customers.
2. Work out loud
One thing I like about Agile is how it has helped redefine team relationships. People chat and draw on white boards, and regular demos provide visibility into progress. If nothing else, it’s helped make development more inclusive and cross-functional.
This is something I try to keep in mind when I consider how to approach usability testing with Agile. More people get work done through face-to-face conversations and phone calls, so it helps to externalize the process and collaborate on usability studies.
This is especially important when it is time to do the testing. Invite people to observe the sessions remotely. Let everyone see how the research is done. This provides the visibility they have gotten used to, but also shows the difference between simply asking them what they want versus observing them in action.
Doing usability testing collaboratively is also an opportunity to help demystify the domain. To make testing happen more regularly, you’ll need advocates, so everything you are doing should feel and look easy. Have face-to-face conversations and coach others by using simple terms, and use techniques others can understand and emulate.
3. Show AND Tell
I recently put dashboard onscreen in front of a team, and the first question asked was along the lines of “So what does this tell us?” It was a reminder that without context, data is just more work. When it comes to reporting, take a page out of Steve Krug’s book and don’t make people think.
Here are 3 things to keep in mind when sharing usability testing results with an Agile team:
- Present – don’t just report. I’ve made the mistake of confusing research data with the storytelling necessary to help an audience understand what the data means. Instead of reading from text-heavy slides, tell a simple story contrasting expected vs. actual user behavior. Keep the findings light and be conversational – even if you’re speaking to 20 people. Your slides should only be a backdrop.
- Have an answer for “So what?” Include some analysis, and commit to recommendations. No one should be left thinking “So what does this mean, what do we do now?” You don’t have to be prescriptive, but you will deliver value if they give the team a clear path forward.
- Let users speak for themselves. Invariably, questions arise about the interpretation so if possible, try to let the customers do some of the talking. If you recorded audio and video, you can bring in a few relevant clips. Seeing analysis directly supported by the customers’ own words adds credibility, and can also help developers feel connected to customers.
Summary
This is specific to the case of “last minute usability testing.” Ideally, it’s wise to ask tough questions much earlier in the process before development has begun in earnest, when it’s cheaper to iterate on designs.
The internet is full of great advice on how to do usability testing in Agile, but my approach is really about first trying to understand people and processes and, to the extent that it doesn’t result in a compromise on user needs, align with them. If your goal is to make usability testing less of a nice-to-have and more of a standard procedure, focus on the variable you can actually control – your approach to the work.
To learn more about testing ideas at any stage of product development, check out some of the resources at Handrailux.com.
If you are looking for help to integrate design and research in an agile framework, consider talking with UX companies that support design and research as part of their process.
About Handrail
We built Handrail to help teams collaborate throughout the entire user research process. Plan, collect, analyze, store, and share your research all in one location. Sign up for a free 30-day trial today.
Great post! A detailed clarification of the requirements is the so-called kick-off - quite long (usually) calls, in which we refine user stories.
The fact is that sometimes initially there is too little information in them to start development - and we, testers, need to immediately articulate exactly what should happen.
Kick-offs allow the tester to understand exactly what the client wants. So the tester finds out, organizes and writes down the requirements for what should be developed.