Section 01
What Are Filter Groups?
Filter Groups is a new feature in TestSynthia that lets you divide a single simulated survey into named demographic segments. Instead of running one panel of 500 anonymous respondents and reading an average, you define exactly who's in each slice:
- Software Developers — ages 25–40, occupation filtered to software development, 20 personas
- Managers — ages 30–55, occupation filtered to management roles, 20 personas
- Female Graduates — ages 22–35, graduate education, female, 15 personas
Each group gets its own filters (age, gender, income, location, education, occupation, target persona) and its own persona count. The panel is the union of the groups — and because groups are disjoint by construction, every persona belongs to exactly one segment. Every response row is tagged with its group, so results can be sliced by segment automatically.
Section 02
Why Segment Splits Beat Panel Averages
The single most common way founders misread survey data is reading the average and missing the disagreement. A purchase-intent score of 3.2 out of 5 can mean everyone is lukewarm — or that one segment is at 4.6 and another at 1.8. Those two worlds demand opposite decisions, and an average destroys the signal that tells you which world you're in.
Illustrative example: identical panel average, opposite conclusions per segment.
This is the same principle behind stratified sampling in traditional research — but traditional research makes you pay per recruit for each quota. With simulated panels, adding a comparison segment costs minutes, not recruiting fees. The economics change what's worth asking: comparing 4–6 segments side by side becomes the default, not the exception.
| Approach | What you see | Typical cost to compare 3 segments | Decision quality |
|---|---|---|---|
| One panel, one average | A single number | 1 × panel cost | Average hides segment conflict |
| One panel, post-hoc slices | Slices only as deep as your data goes | 1 × panel cost | Thin slices = noisy cells |
| Filter Groups | Purpose-built segments with guaranteed sample size each | 1 × panel cost (same total personas) | Every segment has the n you designed for |
Section 03
How A/B Testing With Filter Groups Works
The workflow takes about two minutes inside the survey creation wizard:
- Configure the first group's filters. Pick a panel template or copy demographics from a previous simulation, then adjust age, gender, income, location, education, and occupation for the segment — e.g., software developers aged 25–40.
- Set the persona count for this group (e.g., 20) and hit Save. An AI-generated name like 'Software Developers' is created automatically from the filters — no naming required.
- Clear the filters, configure the next segment. Same flow for 'Managers' or any other comparison group. Each group remembers which panel source it was composed from.
- Launch one simulation. The pipeline filters each group independently against the persona database — Software Developers get their own query with their own limit, then Managers, with each group excluding personas already claimed by earlier groups. No persona appears in two segments.
- Read the comparison. Every response row carries its group tag, so ratings, themes, and objections can be compared group-by-group instead of averaged away.
Input
- Product
- B2B developer tool, $49/mo
- Group A
- Software Developers — 20 personas
- Group B
- Managers — 20 personas
- Questions
- Same questions, same survey
Output
The A/B signal isn't the 52% — it's the 37-point gap between segments, plus each group's stated reasons. Developers buy for themselves; managers need budget approval. Same product, different sale.
Under the hood, the simulation engine runs a separate demographic query per group against the persona database (1M+ US personas with locked attributes), applies each group's own sample limit, and enforces disjointness with an exclusion list so segments can't overlap. This mirrors how quota-based sampling works in panel research — quotas set before fielding, not slices carved afterward.
Section 04
What to A/B Test Across Demographics
Segment comparison is useful for far more than 'does group X like us more?' The highest-value uses we see:
| Test | Segments to compare | The decision it informs |
|---|---|---|
| Price sensitivity | Budget-conscious vs premium-income personas | Which tier to lead with, whether to offer a discount tier |
| Willingness to pay | End users vs budget holders | Who the product must sell to — user-loved tools die without buyer buy-in |
| Message resonance | Technical vs non-technical roles | Which headline and landing-page language to lead with |
| Feature prioritization | Power users vs casual users | Whether a requested feature expands the market or deepens one niche |
| Channel fit | Age brackets (22–30 vs 40–55) | Where to spend the first marketing dollar |
| Geographic expansion | Urban vs suburban vs rural | Which market to enter second |
Section 05
Reading Filter Group Results Honestly
Segment comparisons sharpen conclusions, and like any sharpened instrument they can cut carelessly. Three rules for reading group results:
- Respect sample size per cell. A 15-persona group gives directional signal, not precision. When two groups differ by less than the noise floor of a small cell, treat it as a hypothesis to re-test with more personas, not a verdict. (Our confidence-score methodology — split-half TVD — exists exactly for this; segment-level analyses inherit the same caution.)
- Groups are disjoint, so comparisons are clean. Because each persona belongs to exactly one group, a difference between groups is a real difference in composition — not double-counted respondents. That's the A/B-test property: independent arms.
- The gap is the finding. 'Developers 71%, managers 34%' is more actionable than either number alone. It tells you the product sells bottom-up, that you need a manager-facing pitch, or that your ICP definition needs revisiting — three concrete next steps an average would never surface.
FAQ