The Strength of Great User Testing

User testing is an essential and valuable way to determine if you’re on the right track for what you’re building, but more often the value of the testing falls short of what is hoped to be accomplished.

Usability and user research has steadily gained prominence over the last several decades in software development, and nearly everyone agrees that the insights and information gained is highly valuable to building a great product. As anyone who has worked on a project knows, the effect of getting too close to your work has a subversive and ultimately undermining effect on the quality of the end result. Assumptions made in the moment have a sinister way of becoming concrete facts without the data to support such certainty. Being challenged on your own assumptions is often the wellspring from which true innovation happens; were two divergent ideas can merge to become something bigger and greater.

History is littered with products and services that seemed like great ideas on paper, but the end result was something that the audience didn’t want or ask for. Perhaps the idea simply got away from its originators, or perhaps the inventor or company made something for themselves and not for their customers. It’s easy to have what started as a simple, internal request take on a life of its own and before you know it you’ve changed the core fundamentals of your product and service to meet the needs of the one, rather than the many. User testing helps to solve that problem; it provides as close to a real-world preview as possible while still in the safety of an unreleased product.

But focus groups and user testing can also bring about as much angst as they solve. The event of simply arranging the testing can create additional burdens to the development team, who now have an additional release to plan for… because despite any assurances to the contrary, taking a planned midpoint release with bugs and putting it in front of the public (closed or not) isn’t something most companies will sign off on. The interruption to the schedule (which rarely has padding in it to begin with) in order to make room for the testing is usually unwelcome, even though the team is often pressured to stay positive given the potential benefits. There’s documentation to write, and the fear that the testing will uncover the need for fundamental changes that the product doesn’t have time for.

Then there’s another factor that often creeps into the process: the pressure to make sure the testing confirms the product being built, rather than honestly test the product’s merits. If you’re a product owner, the prospect of having to go back to your management and explain that many of the core features you argued for tested poorly is enough to give even seasoned employees pause. If your company has invested heavily into a product, only to learn that it didn’t test well, it may feel like you just spent more money to learn that the money you’ve been spending was wasted. As a result, I’ve watched many user tests and focus groups be managed aggressively to provide a specific pre-planned result rather than an honest assessment of how the product is received. Thus, the user testing becomes more performance than analysis with the result being validation rather than verification. Of course, self-preservation is an important corporate skill, but in this context the whole effort becomes a waste of a great opportunity to get actual insights into the creation itself.

It is for all of these reasons and many more, the prospect of user research and testing is usually met with as much apprehension as excitement. Where people fall in that range usually has a lot to do with their role in the project… people closely tied to the product team are often far less fond of testing than those responsible for the marketing and sales of it. Still, in the long run the benefits still outweigh the negatives by a huge margin, as long as the testing is entered into with eyes open and the right expectations around the purpose of it.

To have really incredible results with your focus group testing and user research studies, you should aim to have these elements as part of it:

1. The user testing sessions are planned from at the start, and is part of the schedule from the beginning. Make the testing a milestone of the project that is known by the whole team and not a surprise at the last minute or something inserted late in the process.
2. The organizer is someone who isn’t part of the product team. Bring in an external expert resource or a trusted partner from a different team… either way, create some separation from people who may be too close to their work to assess it (or hear feedback) honestly.
3. User testing isn’t a field trip; it’s supposed to be fun, but it shouldn’t be viewed as a break or a party. The best user testing has clear goals and clear outcomes that make the product better and is just as serious as setting the architecture itself.
4. The original, high-level project goals and aims are validated as part of the user testing, as opposed to a specific feature or component. Even if the project itself is massive, it’s still important to keep the overall vision and aims in mind and not allow a segment of the product to be tested in complete isolation.

But most of all… test early. The later in the schedule the user testing is, the less likely that fundamental changes in scope can happen, even if they are clearly needed. Deadlines and schedules are hard to change at the 11^th hour, and learnings are more likely to be seen as annoying than helpful. This means that the engineering team will have more stress to create tangible material earlier, but this isn’t an insurmountable hurdle. Test your design before it’s actually developed into code. Encourage rapid prototyping from your engineers, and test those prototypes. Quick prototypes that demonstrate a core piece of UI or functionality (even if it is “throwaway code”) can be incredibly valuable to test early and inform architecture.

The benefits outweigh the negatives, and the problems are easily managed with modest effort. A great user research study will create better products for the marketplace and will save massive amounts of time and money. Getting real-world insights can be eye opening for an internally focused team and provide perspectives they’ve never had before. As a part of the company’s culture, it can set a great foundation for how products are imagined, built and delivered. If your company isn’t performing user testing, you should.