Constant and ferocious change of technology, platform, delivery, and expectations continue to pose different set of challenges for designers, developers, and UX experts. However, what we take away from usability testing remains constant; as a UX professional, we need to know how to filter, translate, and interpret test results.
REAL LIFE vs. SIMULATION
First of all, we need to acknowledge one obvious fact: the way users behave or react during the test isn’t exactly how they would behave or react in the real world situations. Few reasons why this is so:
- The logistics of user testing typically involve reciprocity. Compensation in the form of some monetary value is offered to participants in return for their effort and contribution. This reciprocal nature affects peoples’ motive, behavior, and conviction. We’d hate to think it does, but that’s human nature.
- User’s actions or inaction during the tests are not linked to personal loss or gain of any kind. Participants are not serious about the choices they make because they do not have to worry about the consequences.
- The knowledge that they are being watched or recorded will affect their behavior. This, again, is human nature. Presence of others affects the way we process information and behave both consciously and subconsciously.
All of these external forces and stimuli significantly influence user response. That is why it’s critical for UX professional to learn how not to take what users say literally. The truly valuable insight into the users’ minds are hidden between the lines.
LITERAL MEANING vs. ACTUAL MEANING
We need to master the art of translating the literal meaning into its actual meaning. A user blurts out “I’d like some sort of instruction here telling me what to do…” Does that mean the user really wants to see an instruction? The worst thing you can do is add an instruction right where the user thinks it should go.
Responses such as “…I’d like to be able to do this here…” may sound simple and solution may quickly come to mind. However, we need to view things in a larger and more coherent context in order to get to the true cause.
ROOT CAUSE vs. SYMPTOMS
When interpreting user’s response it’s important to think about what may have caused such response. What users say is almost always less important than why they said it. Drilling down to the ‘why’ will guide us to the root problem. User’s responses are symptoms. They are not describing the problem. That difference is vital. Ability to distinguish between the symptom and the root cause is singlehandedly the most important skill a UX professional must master. It is the key to avoiding flawed usability testing. Also know that the root cause rarely boils down to a single culprit. Issue in question may have arisen from multiple sources that caused series of problems.
PREFERENCE vs. USABILITY
It is also important to be able to distinguish preference from true usability issue. Any given response from a test participant may be a mixture of personal preference, emotion, and thought process. It’s the trained observation skills and keen understanding of human behavior that effectively identify and extract useful information from such unsorted output.