Combining analytics and qualitative research
Two kinds of evidence
Analytics are quantitative: they show what people do and how many do it. Interviews and usability tests are qualitative: they show why people behave that way. Each has a blind spot, so the strongest decisions use both.
A simple workflow
- Define a question first. For example: “Where do people leave the sign-up process?” Opening a dashboard without a question invites random exploring.
- Find the drop-off. A funnel report lists the steps of a flow and shows how many people continue at each one. A large drop at one step tells you where to look.
- Study that step. Run a few usability sessions or interviews focused on it to learn the reason.
- Change and measure again. Check whether the drop-off improves after the fix.
Triangulate and stay honest
Triangulation means checking a finding against several sources, such as analytics, support requests, interviews and tests. If different sources point the same way, confidence grows. If they disagree, treat the conflict as a question to investigate.
Vanity metrics
Totals such as page views or downloads can rise without anyone getting value. Prefer measures tied to a goal, like task completion.
Small samples
Interview findings explain behaviour but do not tell you how common it is. Use analytics for that.
Good questions to start with
A useful question is specific and tied to a decision. Compare these examples.
Too vague
“How is the website doing?”
Better
“At which step of checkout do most people leave?”
Next, qualitative
“What are people trying to do on that step, and what stops them?”
Break results down by meaningful segments, such as device type or new versus returning visitors, because an overall average can hide a problem that affects only one group.
Privacy considerations
- Tell people what is collected and why, and follow the privacy laws that apply to your users.
- Session recordings can capture personal data. Mask fields such as passwords, payment details and messages.
- Collect only what answers your question, restrict who can view recordings and delete them when they are no longer needed.
Numbers tell you where to look. People tell you what is going on.
This guide was drafted with AI assistance from public standards and references. For the exact wording of a standard, use the sources listed above.