Measuring Product Experience at Scale
Product
Enterprise Cloud Product
Timeline
4 weeks
Methods
Survey & Usability Testing (unmod)
Metrics: SUS; PMF; Task Success; Efficiency
Context & Problem
The team needed a reliable way to measure UX over time, track improvements, and prioritize product decisions.
I led the usability component of a recurring scorecard, partnering with quantitative research to build a scalable mixed-methods system.
The core challenge: how do we measure product experience consistently and make it actionable over time?
Research Goals
- Measure usability, usefulness, and PMF
- Evaluate core workflows
- Identify task barriers
- Track progress across releases
My Role
Owned the usability workstream within the benchmarking system
-
Designed research approach, discussion guide (task scenarios), and recruitment strategy
-
Built and tested unmoderated study in UserZoom
-
Aligned with colleague on analysis & synthesis
-
Led final report creation for both workstreams (qualitative & quantitative)
-
Integrated usability findings with survey insights
Impact
- This program became a primary input for product prioritization and experience improvements
- Increased SUS (14 → 36)
- Increased PMF (19 → 23)
- Identified performance and tool configuration as key drivers
Org impact: Established a repeatable UX measurement system
Reflection
This work represents efforts to integrate qualitative usability testing with quantitative survey data into a unified benchmarking program. While it provided a more complete picture, it also surfaced challenges in aligning methods, timing, and outputs.
My UXR colleage and I coordinated across global time differences, aligning on project timelines, survey and usability questions, and the synthesis of findings into a single, clear narrative for stakeholders.
Although we successfully navigated these challenges, the experience highlighted the need for strong upfront coordination when combining methods to ensure the insights truly work together.
Takeaway: Mixed-methods research is most effective when it’s intentionally designed to connect from the start.