A leadership case study on designing flexibility, reliability, and long-term resilience into a research platform.
Suzy is a market research platform that helps companies create studies and make better business decisions from consumer feedback, while Crowdtap is the consumer community where people answer surveys and share their opinions.
The Problem
For years, the platform relied on one workflow and a single audience source. It produced predictable results until audience availability and quality started fluctuating. Studies became harder to fill, timelines were less reliable, and Ops needed to step in more frequently.
Quality: 25% spam responses following audience quality attacks.
Reliability: Only 21% of surveys filled within three days, down from the 50% benchmark.
Operations: Recurring interventions and rigid screening workflows increased manual work.
Original Model
Audience availability fluctuation
Business Goal
Restore the three-day survey fill rate from 21% to at least the 50% benchmark and keep fulfillment consistent.
My Role
I directed my designer/researcher to:
Synthesize user feedback, stakeholder interviews, and platform audits to uncover workflow dependencies and recurring pain points.
Conduct cross functional workshops bringing audience teams and Ops to surface problems and priorities Prototype solutions to evaluate competing priorities and tradeoffs.
Balance immediate survey reliability improvements with long-term, audience-agnostic experience within limited engineering capacity.
Provide feedback and product design direction, ensuring quality, consistency and solutions aligning with problems.
Research
The existing workflow was built around one audience source. When that source couldn’t reliably fulfill studies, the platform had limited flexibility to bring in additional participants.
Interviews with clients, stakeholders, and internal ops surfaced that the screener was happening too far apart and too early from surveys.
Our research revealed competing priorities across teams:
Audience: Prevent spam and maintain response quality.
Ops: Keep surveys on track and minimize delays.
Clients: Complete surveys quickly without rigid screening requirements.
The main issue: our reliance on a single audience source.
How the team delivered
The solution addressed two parts of the workflow: how clients qualified participants and how Ops managed audience fulfillment.
For Ops — More Control Over Audience Fulfillment
Introduced admin controls that allowed Ops to switch between audience sources when studies were slow to fill.
Enabled manual audience top-ups, with Ops monitoring fill rates daily and adding participants from alternative sources as needed.
Reduced reliance on a single audience source without requiring automated top-up functionality.
For Clients — More Flexible Screening
Integrated screeners directly into the survey creation flow.
Added audience limits and more flexible quota controls to help clients specify who could participate.
Expanded screener question types to include written responses, ratings, and grids.
Streamlined qualification logic to support more specific audience requirements.
Easier setup for audience limits, so teams could control who participates directly while building the survey.
Establish a hierarchy of control to ensure survey and study samples mirror the target population or key consumer segments.
More ways to ask questions, including written responses, ratings, rankings, and grid-style questions.
Screening questions built into the survey, so the right people could be included before continuing.
Simpler rules for showing, hiding, or skipping questions based on someone’s answers.
Tradeoffs
We prioritized internal admin controls over automated audience top-up. This allowed Ops to monitor survey fill rates daily and manually top up underperforming studies by switching audience sources.
We also deferred reusable screener templates to focus engineering capacity on the immediate audience-availability problem.
| Decision | What we deferred | What we gained |
|---|---|---|
| Ship internal admin controls first | Automated audience top-up | Faster Ops intervention when studies were slow to fill |
| Delay reusable screener templates | Reusable audience-screening workflows | Faster delivery against the immediate audience-availability problem |
The outcome
We were able to have:
More predictable study completion, fewer manual interventions, more resilient fulfillment from 21% to 66%
Fulfillment support scaled: 1,500+ client projects serviced through manual, Ops-managed audience top-ups.
Fill reliability recovered: 66% of surveys filled within three days, up from 21% and exceeding the 50% benchmark by 16 percentage points.
Short-term gains
Long-term impact