gail alderson

work/04

Benchmarking UX

Built Microsoft Advertising's first UX health benchmark for a major UI refresh.

Context

what is Microsoft Advertising?

Microsoft Advertising helps marketers launch and manage search campaigns that reach hundreds of millions of people globally.

problem statement

Ahead of the 2019 holiday season, we needed to validate whether the newly refreshed UI was usable for 150k+ advertisers.

old UI (left) vs. new UI (right)
fig. 1old UI (left) vs. new UI (right)

Approach

my roles

  1. UX research — planned research, recruited participants, executed study, analyzed data
  2. Product design — created prototypes, led future work prioritization discussions

research goal

Establish a repeatable quantitative and qualitative UX benchmark for the new UI.

research questions

  • Can users complete critical tasks successfully?
  • Where do users struggle, and how severe are those issues?
  • How confident and satisfied do users feel in the new UI?
  • What SUS score does the experience earn?

Analysis

SUS response

  1. What it is — System Usability Scale (SUS), a 10-question industry-standard usability survey.
  2. Why I used it — Enabled apples-to-apples tracking over time and against competitor products.
example SUS score shown
fig. 2example SUS score shown
click paths (helpful for identifying problematic pages)
fig. 3click paths (helpful for identifying problematic pages)

task analysis

  1. What it is — Task-level analysis framework across success rate, ease, and interaction telemetry.
  2. Why I used it — Made usability, accessibility, and bug patterns easy to identify and communicate.

UI problem matrix

  1. What it is — User-by-problem matrix that prioritizes issues by frequency and impact.
  2. Why I used it — Gave stakeholders a clear, objective backlog for redesign and follow-up studies.
UI problem matrix
fig. 4UI problem matrix

Outcome

Uncovered and addressed 52 usability, accessibility, and responsive design concerns.study now performed annually

findings

  • Exposed a major cross-site reflow failure
  • Informed a redesign of global navigation hierarchy
  • Created a prioritized roadmap of UX fixes and feature opportunities

my impact

  • Established the team's first formal UX health metrics, now reviewed year over year
  • Helped resolve 52 usability, accessibility, and responsive design issues
  • Set the baseline for internal benchmarking and competitive usability comparisons