-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathreferences.html
More file actions
301 lines (292 loc) · 25.6 KB
/
Copy pathreferences.html
File metadata and controls
301 lines (292 loc) · 25.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>References: The Anchorflight Check for Synthetic Users</title>
<meta name="description" content="The Anchorflight Check for Synthetic Users helps product teams know when synthetic users help and when they'll fool you.">
<meta property="og:title" content="The Anchorflight Check for Synthetic Users">
<script src="https://cdn.tailwindcss.com"></script>
<script src="https://unpkg.com/lucide@latest"></script>
<style>
@import url('https://fonts.googleapis.com/css2?family=Geist:wght@300;400;500;600;700&family=Geist+Mono:wght@400;500;600&display=swap');
:root {
--pf-dark:#0F2124; --pf-bg:#f5f4f0; --pf-alt:#eeecea; --pf-card:#fafaf9;
--pf-accent:#3a3aff; --pf-accent-hover:#2d2de0; --pf-accent-dark:#8080ff;
--pf-text:#0d0d0d; --pf-text2:#6b6860;
--pf-success:#16a34a; --pf-danger:#dc2626; --pf-amber:#f59e0b;
--pf-border:rgba(13,13,13,0.10); --pf-border-dark:rgba(255,255,255,0.10);
}
body { font-family:'Geist', system-ui, sans-serif !important; background:var(--pf-bg); color:var(--pf-text); }
.pf-mono { font-family:'Geist Mono', ui-monospace, monospace; }
.pf-display { font-weight:300; letter-spacing:-0.025em; line-height:1.05; }
.pf-card { background:var(--pf-card); border:1px solid var(--pf-border); border-radius:1rem; }
.pf-dark { background:var(--pf-dark); color:rgba(255,255,255,0.92); }
.pf-btn { display:inline-flex; align-items:center; justify-content:center; gap:.5rem; border-radius:9999px; font-size:.875rem; font-weight:500; padding:.75rem 1.5rem; transition:all .18s ease; cursor:pointer; }
.pf-btn-primary { background:var(--pf-accent); color:#fff; }
.pf-btn-primary:hover { background:var(--pf-accent-hover); }
.pf-pill { display:inline-flex; align-items:center; gap:.4rem; font-family:'Geist Mono', monospace; border-radius:9999px; font-size:.72rem; border:1px solid var(--pf-border); color:var(--pf-text2); padding:.35rem .85rem; }
.pf-accent { color:var(--pf-accent); }
/* ---- Global reskin layer: map legacy Tailwind neutrals to the palette.
Leave semantic emerald/amber/rose/indigo/violet alone. ---- */
.bg-white { background-color:var(--pf-card) !important; }
.bg-slate-50, .bg-slate-100, .bg-slate-950, .bg-slate-900 { }
.bg-slate-50, .bg-slate-50\/50, .bg-slate-100 { background-color:var(--pf-alt) !important; }
.text-slate-900, .text-slate-800 { color:var(--pf-text) !important; }
.text-slate-700, .text-slate-600, .text-slate-500, .text-slate-400 { color:var(--pf-text2) !important; }
.border-slate-200, .border-slate-100 { border-color:var(--pf-border) !important; }
.bg-blue-600, .bg-blue-500 { background-color:var(--pf-accent) !important; }
.hover\:bg-blue-500:hover, .hover\:bg-blue-700:hover, .hover\:bg-blue-600:hover { background-color:var(--pf-accent-hover) !important; }
.text-blue-600, .text-blue-700, .text-blue-500 { color:var(--pf-accent) !important; }
.border-blue-500, .border-blue-600, .border-blue-200 { border-color:var(--pf-accent) !important; }
.shadow-sm, .shadow, .shadow-md, .shadow-lg, .shadow-xl { box-shadow:none !important; }
</style>
</head>
<body class="bg-slate-50 text-slate-800 min-h-screen">
<!-- Header -->
<header class="bg-white border-b border-slate-200 sticky top-0 z-50 shadow-sm">
<div class="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-4 flex items-center justify-between gap-4">
<div class="flex items-center gap-2.5">
<span class="w-5 h-5 rounded-md" style="background:var(--pf-accent)"></span><span class="pf-mono text-sm font-semibold tracking-[0.18em]" style="color:var(--pf-text)">ANCHORFLIGHT</span>
</div>
<a href="index.html" class="flex items-center gap-1.5 text-xs font-semibold text-slate-600 hover:text-slate-900 border border-slate-200 hover:border-slate-300 px-3 py-1.5 rounded-lg transition-all">
<i data-lucide="arrow-left" class="w-3.5 h-3.5"></i> Back to App
</a>
</div>
</header>
<main class="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-12 space-y-12">
<!-- Page Header -->
<div>
<div class="flex items-center gap-2 mb-3">
<span class="bg-blue-100 text-blue-800 text-xs font-bold px-3 py-1 rounded-full border border-blue-200 uppercase tracking-wider">Research Basis</span>
</div>
<h1 class="text-3xl font-extrabold text-slate-900 tracking-tight">References</h1>
<p class="text-slate-500 text-sm mt-2 max-w-2xl">The framework and evaluation criteria in this tool are grounded in the following peer-reviewed and industry literature on synthetic users in UX and human-centred design.</p>
</div>
<!-- Table 1 -->
<section>
<div class="mb-4">
<h2 class="text-lg font-bold text-slate-900">Table 1: Key sources on synthetic users in UX / HCD (pros and cons)</h2>
<p class="text-xs text-slate-500 mt-1">A cross-source summary of the main opportunities and risks documented in the literature.</p>
</div>
<div class="overflow-x-auto rounded-2xl border border-slate-200 shadow-sm">
<table class="w-full text-left text-xs border-collapse bg-white">
<thead>
<tr class="bg-slate-900 text-white text-[11px] font-bold uppercase tracking-wider">
<th class="p-3 border-b border-slate-800 min-w-[220px]">Source</th>
<th class="p-3 border-b border-slate-800 w-16">Year</th>
<th class="p-3 border-b border-slate-800 w-36">Type</th>
<th class="p-3 border-b border-slate-800">Main pros / opportunities</th>
<th class="p-3 border-b border-slate-800">Main cons / risks</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-100 text-slate-700">
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Rosala & Moran, NN/g: "Synthetic Users: If, When, and How to Use AI-Generated Research"</td>
<td class="p-3 text-slate-500">2024</td>
<td class="p-3 text-slate-500">Industry UX evaluation</td>
<td class="p-3">Fast synthesis of existing domain knowledge; helps with desk research and early hypothesis generation.</td>
<td class="p-3 text-rose-700">Shallow, idealized responses; unsafe for concept testing, prioritization, or persona creation; must not replace real-user research.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Pehar: "AI as Synthetic Users in Human-Centred Design: A Systematic Review"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Peer-reviewed systematic review</td>
<td class="p-3">Speed and cost reduction in early iterations; supports ideation, persona generation, and large-scale simulations for usability/scenario testing.</td>
<td class="p-3 text-rose-700">Authenticity and variability problems, bias, limited emotional/contextual fidelity, and ethical concerns; should augment, not replace, real users.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Gu et al.: "Synthetic users: insights from designers' interactions with persona-based chatbots"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Peer-reviewed empirical study</td>
<td class="p-3">Allows designers to ask follow-ups, test assumptions, and explore "what-if" questions quickly with interactive personas.</td>
<td class="p-3 text-rose-700">Accuracy doubts, out-of-character responses; treated more as brainstorming partners than ground-truth about users.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">ACM Interactions: pieces on synthetic users and UX challenges</td>
<td class="p-3 text-slate-500">2024–2026</td>
<td class="p-3 text-slate-500">Peer-reviewed commentary</td>
<td class="p-3">Acknowledge efficiency, scalable exploration, and potential diversity gains when synthetic users are carefully instrumented and validated.</td>
<td class="p-3 text-rose-700">Structural biases (over-agreeableness, Western defaults, emotional flatness); synthetic personas can echo assumptions instead of surfacing surprises.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">MeasuringU / LinkedIn: "A Review of Experiments with Synthetic Users"</td>
<td class="p-3 text-slate-500">2026</td>
<td class="p-3 text-slate-500">Methodological review</td>
<td class="p-3">Synthetic users can approximate some usability metrics and replicate coarse patterns of human data on well-structured tasks.</td>
<td class="p-3 text-rose-700">Plausible but wrong data, shallow qualitative output, low variability, failures on complex tasks; unvalidated synthetic findings can mislead.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Koji: "Synthetic Users in Research: Validity, Bias, and When AI Personas Are (and Aren't) Trustworthy"</td>
<td class="p-3 text-slate-500">2026</td>
<td class="p-3 text-slate-500">Industry methodology article</td>
<td class="p-3">Useful for vocabulary familiarization, hypothesis generation, pre-interview rehearsal, and stress-testing research instruments.</td>
<td class="p-3 text-rose-700">Documented sycophancy and rating bias; invalid for decision-grade research (concept tests, prioritization, personas) without real-user validation.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Codexical: "Your AI Research Participants Never Once Disagreed With You"</td>
<td class="p-3 text-slate-500">2026</td>
<td class="p-3 text-slate-500">Industry critique</td>
<td class="p-3">Workflow benefits: instant availability, flexible demographics, and speed for exploratory work and concept sketching.</td>
<td class="p-3 text-rose-700">Over-agreeableness, Western cultural defaults, and emotional flatness produce clean but systematically distorted findings.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">UXArmy: "Synthetic Participants in UX Research: Hype or Future?"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Industry UX tool blog</td>
<td class="p-3">Speed, 24/7 availability, cost savings, scalability, and controlled scenarios for early usability testing and benchmarking.</td>
<td class="p-3 text-rose-700">Lack of true emotion and contextual awareness; risk of misleading data and limited usefulness for exploratory/generative research.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Making Science: "Enhancing UX/UI Research with Synthetic Users"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Agency methodology article</td>
<td class="p-3">Scalability and cost reduction; helpful for early-stage concept and prototype testing and affordance validation.</td>
<td class="p-3 text-rose-700">Lack of real empathy, shallow but superficially accurate insights, dependence on historical data quality; real-user research remains essential.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Holli Downs: "Synthetic Users" (Phosphor discourse review)</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Discourse review / synthesis</td>
<td class="p-3">Efficiency, scale, privacy benefits, and potential data-diversity gains when synthetic users are used carefully.</td>
<td class="p-3 text-rose-700">Reduced authenticity, inability to reproduce emotional/contextual nuance, and overselling as replacements; argues for hybrid models.</td>
</tr>
</tbody>
</table>
</div>
</section>
<!-- Table 2 -->
<section>
<div class="mb-4">
<h2 class="text-lg font-bold text-slate-900">Table 2: Best-practice guidance for using synthetic users</h2>
<p class="text-xs text-slate-500 mt-1">Actionable recommendations distilled from the same body of literature.</p>
</div>
<div class="overflow-x-auto rounded-2xl border border-slate-200 shadow-sm">
<table class="w-full text-left text-xs border-collapse bg-white">
<thead>
<tr class="bg-slate-900 text-white text-[11px] font-bold uppercase tracking-wider">
<th class="p-3 border-b border-slate-800 min-w-[220px]">Source</th>
<th class="p-3 border-b border-slate-800 w-16">Year</th>
<th class="p-3 border-b border-slate-800 w-36">Type</th>
<th class="p-3 border-b border-slate-800">Key best-practice themes / guidance</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-100 text-slate-700">
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Rosala & Moran, NN/g: "Synthetic Users: If, When, and How to Use AI-Generated Research"</td>
<td class="p-3 text-slate-500">2024</td>
<td class="p-3 text-slate-500">Industry UX evaluation</td>
<td class="p-3">Treat synthetic output as desk research and hypothesis fodder only; never present as "users said"; avoid using for concept validation, prioritization, or persona creation; be cautious in low-maturity orgs.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Pehar: "AI as Synthetic Users in Human-Centred Design: A Systematic Review"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Systematic academic review</td>
<td class="p-3">Adopt a hybrid HCD model where synthetic users are used in early stages but key decisions are validated with real users; document where AI replaces or augments user input; calibrate and validate simulations; use for scenario coverage rather than everyday feedback.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Gu et al.: "Synthetic users: insights from designers' interactions with persona-based chatbots"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Empirical study</td>
<td class="p-3">Position chat-based synthetic personas as interactive extensions of personas, not replacements; use to test assumptions and explore ideas; balance consistency and variation in behavior; treat them as creative partners, not authoritative users.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">ACM Interactions: pieces on synthetic UX research</td>
<td class="p-3 text-slate-500">2024–2026</td>
<td class="p-3 text-slate-500">Peer-reviewed commentary</td>
<td class="p-3">Make structural biases explicit in method sections; build validation loops comparing synthetic and human findings; use synthetic research only where later human data can falsify or confirm it; avoid letting synthetic personas stand in for ethnographic evidence.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Koji: "Synthetic Users in Research: Validity, Bias, and When AI Personas Are (and Aren't) Trustworthy"</td>
<td class="p-3 text-slate-500">2026</td>
<td class="p-3 text-slate-500">Industry methodology</td>
<td class="p-3">Use synthetic users for vocabulary familiarization, hypothesis generation, and rehearsing guides; never for roadmap/feature decisions; follow a two-day hybrid workflow (Day 1 synthetic, Day 2 real interviews); label outputs as hypotheses requiring validation.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Codexical: "Your AI Research Participants Never Once Disagreed With You"</td>
<td class="p-3 text-slate-500">2026</td>
<td class="p-3 text-slate-500">Industry critique</td>
<td class="p-3">Assume structural over-agreeableness; run small real-user samples alongside synthetic studies; treat divergence as a signal; pre-define which findings must be falsifiable with real users before shipping.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">UXArmy: "Synthetic Participants in UX Research: Hype or Future?"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">UX tool blog</td>
<td class="p-3">Use synthetic participants for early, task-based usability tests and A/B experiments; avoid for discovery interviews or emotion-heavy work; clearly communicate limitations to stakeholders; always validate critical insights with real participants.</td>
</tr>
<tr class="hover:bg-slate-50 bg-slate-50/50">
<td class="p-3 font-medium text-slate-800">Making Science: "Enhancing UX/UI Research with Synthetic Users"</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Agency article</td>
<td class="p-3">Integrate synthetic users into multi-layered models with market, behavioral, and real user data; use for early exploration and prototype validation; maintain real-user research as the foundation of design strategy.</td>
</tr>
<tr class="hover:bg-slate-50">
<td class="p-3 font-medium text-slate-800">Holli Downs: "Synthetic Users" (Phosphor discourse review)</td>
<td class="p-3 text-slate-500">2025</td>
<td class="p-3 text-slate-500">Discourse synthesis</td>
<td class="p-3">Advocate hybrid models combining AI scale with human depth; emphasize transparent reporting of where synthetic vs. real data are used; encourage case-study sharing to converge on community best practices.</td>
</tr>
</tbody>
</table>
</div>
</section>
<!-- Back nav -->
<div class="flex justify-center">
<a href="index.html" class="flex items-center gap-1.5 bg-slate-900 hover:bg-slate-800 text-white px-5 py-2.5 rounded-lg transition-all font-semibold text-sm">
<i data-lucide="arrow-left" class="w-4 h-4"></i> Return to App
</a>
</div>
</main>
<footer class="pf-dark mt-16">
<div class="max-w-6xl mx-auto px-6 md:px-10 py-14">
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-12 gap-10">
<div class="lg:col-span-5">
<div class="flex items-center gap-2.5">
<span class="w-5 h-5 rounded-md" style="background:var(--pf-accent)"></span>
<span class="pf-mono text-sm font-semibold tracking-[0.18em] text-white">ANCHORFLIGHT</span>
</div>
<p class="text-sm text-white/50 mt-4 max-w-xs leading-relaxed">The Anchorflight Check for Synthetic Users: know when to trust AI, and when to send in the humans.</p>
</div>
<div class="lg:col-span-3">
<p class="pf-mono text-[11px] tracking-widest text-white/40">PAGES</p>
<div class="mt-5 flex flex-col gap-3 items-start">
<a href="index.html" class="text-sm text-white/80 hover:text-white transition-colors">Home</a>
<a href="index.html#checkpoint" class="text-sm text-white/80 hover:text-white transition-colors">Active Checkpoint</a>
<a href="index.html#wizard" class="text-sm text-white/80 hover:text-white transition-colors">Cohort Due Diligence</a>
<a href="index.html#map" class="text-sm text-white/80 hover:text-white transition-colors">The 8 Gate Anchorflight Framework</a>
</div>
</div>
<div class="lg:col-span-4">
<p class="pf-mono text-[11px] tracking-widest text-white/40">RESOURCES</p>
<div class="mt-5 flex flex-col gap-3 items-start">
<a href="index.html#sandbox" class="text-sm text-white/80 hover:text-white transition-colors">Synthetic Persona Prompt Architect</a>
<a href="index.html#usability" class="text-sm text-white/80 hover:text-white transition-colors">Usability Tester</a>
<a href="index.html#rationale" class="text-sm text-white/80 hover:text-white transition-colors">Lifecycle Rationale</a>
<a href="index.html" class="text-sm text-white/80 hover:text-white transition-colors">How to Build Synthetic Users</a>
<a href="index.html" class="text-sm text-white/80 hover:text-white transition-colors">Usability Evaluation Guide</a>
</div>
</div>
</div>
<p class="text-center text-white/50 text-sm mt-16 max-w-2xl mx-auto leading-relaxed">Always align synthetic persona configurations with physical customer telemetry, support tickets, and direct feedback channels.</p>
<div class="flex flex-wrap items-center justify-center gap-x-3 gap-y-2 mt-6 text-xs text-white/50">
<a href="references.html" class="hover:text-white underline underline-offset-2 transition-colors">References</a>
<span class="text-white/20">·</span>
<a href="https://www.apache.org/licenses/LICENSE-2.0" target="_blank" class="hover:text-white underline underline-offset-2 transition-colors">Apache-2.0</a>
<span class="text-white/20">·</span>
<a href="https://github.com/vivistar/synth" target="_blank" class="hover:text-white underline underline-offset-2 transition-colors">github.com/vivistar/synth</a>
<span class="text-white/20">·</span>
<a href="https://github.com/vivistar/synth/discussions" target="_blank" class="hover:text-white underline underline-offset-2 transition-colors">Feedback</a>
</div>
<div class="mt-10 pt-6 flex flex-col sm:flex-row items-center justify-between gap-3" style="border-top:1px solid var(--pf-border-dark)">
<p class="pf-mono text-[11px] text-white/40">Anchorflight · Powered by OpenRouter · <span id="footer-year"></span></p>
<p class="text-xs text-white/40">© <span class="pf-year"></span> Terry M. Patterson · Licensed under Apache-2.0</p>
<p class="pf-mono text-[11px] text-white/40 flex items-center gap-1.5"><span class="w-1.5 h-1.5 rounded-full" style="background:var(--pf-success)"></span> All systems nominal</p>
</div>
</div>
</footer>
<script>
lucide.createIcons();
document.querySelectorAll('#footer-year, .pf-year').forEach(function(el){ el.textContent = new Date().getFullYear(); });
</script>
</body>
</html>