Design Learning Systems

A reusable skill and methodology for AI systems and agents to design, review, and diagnose robust learning systems—from the first baseline to production operations.

It helps choose among supervised learning, imitation learning, reinforcement learning, offline RL, self-play, and hybrid strategies; design independent rewards and evaluations; control experiments; detect pathological behavior; and plan safety, promotion, and rollback.

The methodology is platform- and domain-general. It can guide AI assistants, autonomous agents, research workflows, and engineering teams across robotics, recommendation, classification, forecasting, language, vision, simulation, games, and many other domains. Chess is only one possible example.

What it delivers

• An operational problem definition and explicitly prohibited behaviors.
• Required baselines and a data strategy.
• A recommended learning regime and model architecture.
• Loss, reward, curriculum, and exploration design.
• Defenses against reward hacking and specification gaming.
• Reproducible governance with seeds, versions, hashes, and checkpoints.
• Ablations, confidence intervals, and robustness tests.
• Training, data, policy, and operational observability.
• Containment, integrity tests, and incident classification.
• Promotion gates, progressive deployment, monitoring, and rollback.

Installation

Windows PowerShell

git clone https://github.com/perepepeu/design-learning-agents.git
Copy-Item -Recurse -Force `
  .\design-learning-agents\design-learning-systems `
  "$HOME\.codex\skills\design-learning-systems"

macOS or Linux

git clone https://github.com/perepepeu/design-learning-agents.git
cp -R design-learning-agents/design-learning-systems \
  "${CODEX_HOME:-$HOME/.codex}/skills/design-learning-systems"

Restart or reload Codex if the skill does not appear immediately.

Usage

Invoke the skill explicitly:

$design-learning-systems Design an agent that learns to control warehouse
inventory without increasing stockouts or waste.

More examples:

$design-learning-systems Compare imitation learning, offline RL, and online RL
for a robot.

$design-learning-systems Review this reward function and find shortcuts the
agent could exploit.

$design-learning-systems Create a reproducible protocol for comparing two
recommendation models with multiple seeds and regression tests.

$design-learning-systems Diagnose why reward increased while the agent became
worse on held-out scenarios.

Method

The core workflow has nine stages:

1. Define the objective, decisions, constraints, and risks.
2. Establish baselines and the data strategy.
3. Select the learning regime and architecture.
4. Separate the real objective, training signal, and evaluation.
5. Freeze governance and ablations before seeing results.
6. Instrument behavioral and operational observability.
7. Evaluate with multiple seeds, uncertainty, and held-out scenarios.
8. Design containment and environment-integrity controls.
9. Promote gradually, monitor continuously, and retain rollback.

Structure

design-learning-systems/
├── SKILL.md
├── agents/
│   └── openai.yaml
├── assets/
│   ├── experiment-manifest.json
│   └── learning-system-plan.md
├── references/
│   ├── problem-and-learning-regimes.md
│   ├── model-selection.md
│   ├── reward-and-curriculum.md
│   ├── experimental-governance.md
│   ├── evaluation-and-ablations.md
│   ├── observability-and-failure-modes.md
│   ├── safety-and-alignment.md
│   ├── deployment-and-monitoring.md
│   └── example-applications.md
└── scripts/
    ├── validate_experiment_manifest.py
    └── compare_experiments.py

SKILL.md contains the decision workflow. References are loaded only when relevant, keeping context lean.

Included tools

Validate an experiment manifest

Copy assets/experiment-manifest.json, fill in the fields, and run:

python design-learning-systems/scripts/validate_experiment_manifest.py \
  path/to/experiment-manifest.json --strict

The validator checks minimum reproducibility fields, duplicate seeds, confidence levels, and unresolved placeholders.

Compare a baseline and candidate

Prepare a CSV file:

variant,seed,metric
baseline,101,0.62
candidate,101,0.67
baseline,202,0.64
candidate,202,0.70

Run:

python design-learning-systems/scripts/compare_experiments.py results.csv \
  --baseline baseline --candidate candidate

The script reports descriptive statistics, the paired delta by seed, and a bootstrap interval. It does not replace domain-specific evaluation.

Project principles

• Never claim progress from one seed or a cherry-picked best run.
• Never confuse training reward with real success.
• Never add complexity without a baseline and a testable hypothesis.
• Never hide regressions behind an aggregate mean.
• Never compensate for an environment bug by distorting model behavior.
• Never deploy without containment, telemetry, and a tested rollback path.

Development

See CONTRIBUTING.md to set up, validate, and propose changes. Security issues must follow SECURITY.md, not a public issue.

License

Released under the MIT License.