In the course of my career I’ve been fortunate enough to have attended numerous usability test sessions. I had chances to observe many varieties from the most entertaining to the gawdawful boring, a fifty-something man operating a mouse with both of his hands to a tech-savvy who checks the view source.
Technology and devices have come a long way and continue to challenge designers, developers, and UX experts. However, what we take away from the usability testing remains constant; as an user experience professional, we need to know how to filter, translate, and interpret test results.
REAL LIFE vs. SIMULATION
First of all, let’s establish one obvious fact; how users behave or react during the test isn’t exactly how they would behave or react in the real world situations. In addition to the fact that most participants are after some cash or gift as a compensation, whatever they do during the test has nothing to do with their personal loss or gain. They are not really serious about the choices they’re making. Knowing that they’re playing along a fake scenario and not having to worry about consequences do affect users’ response. That is why it is critical for UX professional to learn how not to take what users say literally. The truly valuable insight into the users’ minds are hidden between the lines.
LITERAL MEANING vs. ACTUAL MEANING
We need to master the art of translating the literal meaning into its actual meaning. A user blurts out “I’d like some sort of instruction here telling me what to do…” Does that mean the user really wants to see an instruction? The worst thing you can do is add an instruction right where the user thinks it should go. There are many ways to guide or nudge the user to intended path without extraneous text or instructions.
ROOT CAUSE vs. SYMPTOMS
When interpreting user’s response it’s important to think about what may have caused such response. What users say is almost always less important than why they said it. Drilling down to the ‘why’ will guide us to the root problem. User’s responses are symptoms. They are not describing the problem. That difference is vital. Ability to distinguish between the symptom and the root cause is singlehandedly the most important skill a UX professional must master. Knowing that one vital difference is the key in avoiding flawed usability testing. Also know that the root cause rarely boils down to a single culprit. Issue in question may have arisen from multiple sources that caused series of problems.
PREFERENCE vs. USABILITY
It is also important to be able to distinguish preference from true usability issue. Any given response from a test participant may be a mixture of personal preference, emotion, and thought process. It’s the trained observation skills and keen understanding of human behavior that effectively identify and extract useful information from such unsorted output.
We also need to remember that usability does not have direct one to one relationship with how attractive something is. There is a correlation to some degree, but it is quite possible for an ugly site or an app to have strikingly easy to navigate interface and clearly defined user journey as it is possible for visually attractive site to fail miserably on all accounts.