Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/ISSUE_TEMPLATE/independent-eval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
name: Independent eval run
about: Submit or coordinate a third-party DesignOS benchmark run
title: "[eval] Run 00X — "
labels: eval, evidence
---

## Run summary
<!-- Model, agent surface, brief set, and whether the result was positive/mixed/negative. -->

## Protocol
- [ ] Same model/settings in control and treatment
- [ ] Same brief text in both arms
- [ ] No follow-up steering unless declared
- [ ] Outputs randomized before judging
- [ ] `evals/judge-prompt.md` used for blind scoring
- [ ] Validator raw output attached
- [ ] Raw generated artifacts attached or linked

## Report
<!-- Paste `evals/RUN_TEMPLATE.md` here or link to the completed file. -->

## What should change because of this?
<!-- README claim, module rule, validator, docs caveat, or no change yet. -->
26 changes: 26 additions & 0 deletions .github/ISSUE_TEMPLATE/showcase-submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
name: Showcase submission
about: Submit a real project built with DesignOS
title: "[showcase] "
labels: showcase
---

## Project
- Live URL:
- Repository or case-study URL:
- Sector/type:

## What DesignOS changed
<!-- One honest sentence: what did it catch, enforce, improve, or fail to improve? -->

## Proof
- [ ] The page/app is publicly reachable
- [ ] The claim does not invent users, metrics, testimonials, or badges
- [ ] You are allowed to share this work
- [ ] You are comfortable with the project being linked from `SHOWCASE.md`

## Suggested row

```markdown
| [Project name](https://live-url) | sector/type | one honest sentence on what DesignOS changed |
```
3 changes: 3 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@
- [ ] Cross-references resolve (reference check script run)
- [ ] Kernel routing table updated (if a module was added)
- [ ] README structure tree updated (if the tree changed)
- [ ] `npm test` run, or reason stated
- [ ] `npm run validate` run, or reason stated
- [ ] Any proof/eval claim names who ran it and links raw outputs
18 changes: 10 additions & 8 deletions LAUNCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,24 @@ Stars come from distribution × proof × timing — not from file count. This is
operational plan for taking DesignOS worldwide. Work it top to bottom; check items off
in your fork.

## Phase 0 — Pre-flight (do BEFORE any public link exists)
- [ ] Replace every `USER` placeholder (README, website, package.json repo fields)
- [ ] Push to GitHub; enable Pages on `website/` (Settings → Pages → /website folder or
a `gh-pages` action); verify the live URL on mobile
## Phase 0 — Pre-flight (do BEFORE the big public push)
- [x] Replace every `USER` placeholder (README, website, package.json repo fields)
- [x] Push to GitHub; enable Pages via GitHub Actions; verify the live URL on mobile.
Last checked: <https://ardamoustafa1.github.io/DesignOS/website/> returned HTTP
200 on 2026-07-17.
- [ ] Repo polish: description line ("Design intelligence OS for AI coding agents —
one prompt in, Stripe-grade UI out"), topics (`claude`, `claude-code`, `cursor`,
`ai-agents`, `design-systems`, `prompt-engineering`, `ui-design`), social-preview
image uploaded — `press/social-preview.png` (1024×1024, dark, the boot-log
image uploaded — `press/social-preview.png` (1280×640, dark, the boot-log
terminal as the visual) is ready at Settings → Social preview → Edit
- [ ] Run the full validator suite + click every README link
- [ ] **Record the 30-second GIF** (the single highest-leverage asset): screen capture
of a real session — prompt in → boot log → modules routing → scorecard → the
showcase page. Put it at the top of the README. Tools: any recorder + gifski.
No GIF = half the launch.
- [ ] Run `evals/` on at least one model pair and put the numbers in the README —
"measured, not vibed" is the differentiator claim; back it before HN asks
- [ ] Get Run 003 from an external user with [`evals/RUN_TEMPLATE.md`](evals/RUN_TEMPLATE.md)
attached — "measured, not vibed" is the differentiator claim; maintainer-only
numbers are useful instrumentation, not proof

## Phase 1 — Seeding (1–2 weeks before the spike)
- [ ] Soft-share in 3–5 niche communities where feedback > reach: Claude Code Discord,
Expand Down Expand Up @@ -62,7 +64,7 @@ in your fork.
each has a submission path; the GIF + one-paragraph pitch
- [ ] Respond to every issue within 24h for the first month — early responsiveness
converts visitors to contributors; slow repos die in the trough
- [ ] Ship a visible v1.3/v1.4 within 3 weeks of launch (the changelog IS marketing —
- [ ] Ship a visible v2.1 within 3 weeks of launch (the changelog IS marketing —
`patterns/changelog.md`, applied to ourselves)

## The message discipline (all channels, one voice)
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

[![GitHub stars](https://img.shields.io/github/stars/ardamoustafa1/DesignOS?style=flat&color=10b981)](https://github.com/ardamoustafa1/DesignOS/stargazers)
[![License: MIT](https://img.shields.io/badge/License-MIT-black.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-2.0.1-blue.svg)](CHANGELOG.md)
[![Version](https://img.shields.io/badge/version-2.0.2-blue.svg)](CHANGELOG.md)
[![Live proof](https://github.com/ardamoustafa1/DesignOS/actions/workflows/proof.yml/badge.svg)](https://github.com/ardamoustafa1/DesignOS/actions/workflows/proof.yml)
[![Validate](https://github.com/ardamoustafa1/DesignOS/actions/workflows/validate.yml/badge.svg)](https://github.com/ardamoustafa1/DesignOS/actions/workflows/validate.yml)
[![WCAG 2.2 AA](https://img.shields.io/badge/A11y-WCAG%202.2%20AA-2ea44f.svg)](checklists/accessibility.md)
Expand Down Expand Up @@ -66,7 +66,7 @@ The agent stops thinking *"I'll make a button"* and starts thinking
**We're trying to measure this, not just claim it.** Below is a maintainer-run sanity
check — **not independent validation** ([full caveats](evals/RESULTS.md)) — showing the
validators catch what they're supposed to catch. The eval slot that would actually prove
the claim is open and unfilled; [run it and PR your numbers](evals/RESULTS.md#run-002--your-model-your-judge-pending--this-is-the-one-that-matters).
the claim is open and unfilled; [run it and PR your numbers](evals/RESULTS.md#run-003--your-model-your-judge-the-one-that-matters-most).

| Mechanical check (same brief) | Default-style output | Through DesignOS |
|---|---:|---:|
Expand Down Expand Up @@ -231,8 +231,9 @@ Every page has its decision paper-trail:
miss, an unverified metric, a CSS cascade bug found at 577px).

🔬 **[Measure it yourself](evals/README.md)** — 10 fixed briefs, a paste-ready
[neutral judge prompt](evals/judge-prompt.md), validators as the objective floor, and
[published results](evals/RESULTS.md) with the caveats stated plainly.
[neutral judge prompt](evals/judge-prompt.md), validators as the objective floor, a
[run report template](evals/RUN_TEMPLATE.md), and [published results](evals/RESULTS.md)
with the caveats stated plainly.

🌐 **[The website](website/index.html)** — designed by the system it documents.
View source: every value resolves to a token.
Expand Down Expand Up @@ -278,7 +279,7 @@ Under threshold → back to the loop with specific objections, *before you ever
- 🏗 **[Add your work to the Showcase](SHOWCASE.md)** — and take the
[![badge](https://img.shields.io/badge/designed%20with-DesignOS-10b981)](SHOWCASE.md) with you
- 🥊 **[Challenge a rule](CONTRIBUTING.md)** — every rule stands on evidence; bring yours
- 🔍 **[Run the missing independent eval](evals/RESULTS.md)** — the single highest-value
- 🔍 **[Run the missing independent eval](evals/RUN_TEMPLATE.md)** — the single highest-value
contribution available right now
- 💬 **[Discussions](DISCUSSIONS.md)** — questions, show-and-tell, module ideas (setup guide; enable in repo Settings)
- 🗺 **[Roadmap](ROADMAP.md)** · 🚀 **[Launch playbook](LAUNCH.md)** · 🛡 **[Security](SECURITY.md)** · 📰 **[Press kit](press/README.md)** · ⚠️ **[Known limitations](LIMITATIONS.md)** · 🏢 **[Enterprise](ENTERPRISE.md)** · ⚖️ **[Governance](GOVERNANCE.md)**
Expand Down
7 changes: 7 additions & 0 deletions evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,10 @@ After editing modules: re-run the affected briefs (a pricing-module change → b
and 8). Score deltas below the noise floor are fine; systematic drops are a
rule-challenge issue (`CONTRIBUTING.md`) with data attached — exactly the evidence
standard we ask of contributors.

## Submitting Run 003+

Use [`RUN_TEMPLATE.md`](RUN_TEMPLATE.md) for independent reports. The template is
intentionally specific: model versions, agent surface, prompt text, raw validator output,
judge-pass medians, and negative findings all belong in the PR. A mixed or negative run
is useful evidence; a perfect-looking report with no raw outputs is marketing, not eval.
77 changes: 77 additions & 0 deletions evals/RUN_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Independent Eval Run Template

Copy this file into a PR section or attach it as `evals/runs/run-00X.md`.
Reports can be positive, mixed, or negative. The only bad report is one that hides
conditions or raw outputs.

## Run metadata

| Field | Value |
|---|---|
| Run ID | Run 003 |
| Date | YYYY-MM-DD |
| Runner | name / GitHub handle |
| Agent surface | Claude Code / Cursor / Copilot / Windsurf / Cline / Aider / other |
| Model(s) | exact model names and versions, if visible |
| OS / browser | environment details |
| Brief set | all 10 / declared subset |
| Time budget | per arm |
| Follow-up steering | none / explain deviation |

## Installation notes

- Control arm setup:
- Treatment arm setup:
- DesignOS commit SHA:
- Any local modifications:

## Validator results

Run the same commands for both arms and paste the raw output or a link to the raw log.

```bash
node validators/check-drift.js path/to/control-output.html
node validators/check-a11y-basics.js path/to/control-output.html
node validators/check-drift.js path/to/treatment-output.html
node validators/check-a11y-basics.js path/to/treatment-output.html
```

| Brief | Arm | Drift findings | A11y findings | Contrast notes |
|---|---|---:|---:|---|
| B-001 | Control | | | |
| B-001 | Treatment | | | |

## Blind judge results

Use `evals/judge-prompt.md`. Run three judge passes when using a model judge and report
medians, not just the best pass.

| Brief | Arm | Hierarchy | Craft | Accessibility | States | Distinctiveness | Fitness | Median total |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| B-001 | A/B label | | | | | | | |

## Raw artifacts

- Control outputs:
- Treatment outputs:
- Judge transcripts:
- Validator logs:

## Findings

### Where DesignOS helped

-

### Where DesignOS hurt or added cost

-

### Ambiguous results

-

## Conclusion

State the result in one honest paragraph. Avoid universal claims. This run measures
this model, this surface, this brief set, under these conditions.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "designos",
"version": "2.0.2",
"description": "The Design Intelligence Operating System for AI coding agents — Claude, Fable, Cursor, Copilot.",
"description": "The Design Intelligence Operating System for AI coding agents — Claude, Cursor, Copilot, Windsurf, Cline, Aider.",
"author": "Arda Moustafa (https://github.com/ardamoustafa1)",
"repository": {
"type": "git",
Expand Down
Loading