Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 122 additions & 16 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,33 +3,139 @@ name: Release
on:
push:
tags:
- "v*"
- "v[0-9]*"

permissions:
contents: read

jobs:
build-and-publish:
name: Build & Publish to PyPI
verify-and-build:
name: Verify & Build Release Artifacts
runs-on: ubuntu-latest
permissions:
id-token: write # For PyPI trusted publishing
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Require tag at current main
shell: bash
run: |
git fetch origin main
test "$GITHUB_SHA" = "$(git rev-parse origin/main)"

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: "3.11"

- name: Install build tools
run: pip install build
- name: Install release dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[dev]" build hatchling twine

- name: Build package
run: python -m build
- name: Build Python distributions and Agent plugin bundle
shell: bash
run: |
python -m build --sdist --wheel --outdir dist
python scripts/build_plugin_bundle.py \
--tag "$GITHUB_REF_NAME" \
--output-dir release-assets
python -m twine check dist/*
(
cd dist
sha256sum *
) > ../release-assets/SHA256SUMS
(
cd release-assets
sha256sum celltypepilot-plugin-*.zip
) >> SHA256SUMS

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
- name: Run release verification suite
run: |
ruff check src/ tests/ scripts/build_plugin_bundle.py
ruff format --check src/ tests/ scripts/build_plugin_bundle.py
pytest --cov=celltypepilot --cov-report=term-missing

- name: Smoke-test installed wheel and plugin bundle
shell: bash
run: |
python -m venv --system-site-packages .release-smoke
.release-smoke/bin/python -m pip install --no-deps --force-reinstall dist/*.whl
.release-smoke/bin/python -c \
"import celltypepilot; assert celltypepilot.__version__ == '${GITHUB_REF_NAME#v}'"
.release-smoke/bin/celltypepilot --version
.release-smoke/bin/celltypepilot doctor --json

mkdir plugin-smoke
python -m zipfile -e release-assets/celltypepilot-plugin-*.zip plugin-smoke
.release-smoke/bin/python -m pip install \
--no-build-isolation --no-deps --force-reinstall \
plugin-smoke/celltypepilot-plugin-*
.release-smoke/bin/celltypepilot --version

- name: Upload Python distributions
uses: actions/upload-artifact@v7
with:
name: python-distributions
path: dist/*
if-no-files-found: error
retention-days: 7

- name: Upload Agent plugin bundle
uses: actions/upload-artifact@v7
with:
name: agent-plugin-bundle
path: release-assets/*
if-no-files-found: error
retention-days: 7

github-release:
name: Publish GitHub Plugin Release
needs: verify-and-build
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download Python distributions
uses: actions/download-artifact@v8
with:
name: python-distributions
path: release-assets/python

- name: Create GitHub Release
uses: softprops/action-gh-release@v2
- name: Download Agent plugin bundle
uses: actions/download-artifact@v8
with:
name: agent-plugin-bundle
path: release-assets/plugin

- name: Create GitHub Release before PyPI publication
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
files: dist/*
prerelease: ${{ startsWith(github.ref_name, 'v0.') }}
fail_on_unmatched_files: true
files: |
release-assets/python/*
release-assets/plugin/*

pypi-publish:
name: Publish Python Backend to PyPI
needs: github-release
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/celltypepilot
permissions:
id-token: write
steps:
- name: Download verified Python distributions
uses: actions/download-artifact@v8
with:
name: python-distributions
path: dist

- name: Publish backend with Trusted Publishing
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/
print-hash: true
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ __pycache__/
*.egg-info/
dist/
build/
release-assets/
plugin-smoke/
.release-smoke/
.eggs/
*.egg

Expand Down
20 changes: 11 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,19 @@ celltypepilot doctor
If core dependencies are missing, install them:

```bash
pip install -e .
pip install celltypepilot
```

For a repository checkout used in development, `pip install -e .` remains appropriate.

CellTypePilot runs on CPU, requires Python >= 3.10, scanpy, anndata, and matplotlib.
No MCP servers, no pixi, no conda environment needed for the basic path.

Optional extras:
- `pip install -e ".[web]"` — Web Inspector (Flask-based interactive review panel)
- `pip install -e ".[mcp]"` — Native local CellTypePilot MCP facade for Agent hosts
- `pip install -e ".[seurat]"` — Seurat .rds conversion (requires rpy2 or R)
- `pip install -e ".[all]"` — All optional features
- `pip install "celltypepilot[web]"` — Web Inspector (Flask-based interactive review panel)
- `pip install "celltypepilot[mcp]"` — Native local CellTypePilot MCP facade for Agent hosts
- `pip install "celltypepilot[seurat]"` — Seurat .rds conversion (requires rpy2 or R)
- `pip install "celltypepilot[all]"` — All optional features

## Workflow — Four Stages

Expand Down Expand Up @@ -176,7 +178,7 @@ All commands support `--json` for structured output.

## Supported tissues

Built-in Marker Knowledge Graph (MKG mkg-2026.08) covers:
Built-in Marker Knowledge Graph (MKG mkg-2026.08.1) covers:
- **Blood/PBMC**: T cells (CD4/CD8/naive/memory/Treg/Th1/Th17), B cells, NK cells, monocytes, DCs, platelets
- **Lung**: alveolar macrophages, AT1/AT2, ciliated, club, goblet, basal
- **Liver**: hepatocytes, Kupffer, cholangiocytes, endothelial, stellate
Expand All @@ -189,7 +191,7 @@ Built-in Marker Knowledge Graph (MKG mkg-2026.08) covers:
- **Skeletal muscle**: myofibers, satellite cells, FAPs
- **General**: endothelial, pericytes, fibroblasts, macrophages, mast cells, epithelial

Premium atlas (requires academic/commercial license):
Bundled extended atlas (historical directory name `premium`, MIT-licensed and open to all users):
- **Tumor microenvironment**: TAMs, CAFs, Tregs, exhausted T cells, MDSCs, malignant cells
- **Developing brain**: radial glia, intermediate progenitors, migrating neurons
- **Inflamed tissue**: activated fibroblasts, M1/M2 macrophages
Expand Down Expand Up @@ -258,7 +260,7 @@ celltypepilot/
│ ├── constants.py # Thresholds, species/tissue constants
│ ├── data/
│ │ ├── marker_atlas.json # Built-in marker knowledge graph (80+ types)
│ │ ├── packs/premium/ # First-party premium pack (academic license)
│ │ ├── packs/premium/ # MIT first-party extended pack (legacy name)
│ │ └── state_atlas.json # Versioned exploratory cell-state modules
│ ├── templates/ # Jinja2 templates (HTML report, web dashboard)
│ ├── marker_scorer.py # DE + marker overlap scoring
Expand All @@ -269,7 +271,7 @@ celltypepilot/
│ ├── visualizer.py # UMAP, dotplot, confidence figures
│ ├── web_inspector.py # Flask web review panel
│ ├── literature.py # PubMed literature validation
│ ├── license_manager.py # Tiered license system
│ ├── license_manager.py # Legacy service-entitlement compatibility
│ ├── provenance.py # manifest.json generation
│ ├── reporter.py # HTML report + methodology text
│ └── doctor.py # Environment check
Expand Down
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Changelog

All notable changes to CellTypePilot are documented here. The project follows
[Semantic Versioning](https://semver.org/). Release claims remain bounded by the
validation scope recorded in the repository and generated manifests.

## [0.3.0] - 2026-08-10

### Added

- Four-step Agent golden workflow with the `celltypepilot.agent-decision.v1` protocol.
- Actionable evidence gaps for `Unknown` decisions and contrastive top-two evidence.
- Independent Identity, State, and Novelty/OOD review axes with human sign-off boundaries.
- Governed context packs, data-only extension packs, atlas governance, and evidence promotion.
- Donor-aware benchmark infrastructure, comparator adapters, resumable execution, and QC diagnostics.
- Native MCP facade, Web Review audit artifacts, and dual Codex/Claude plugin manifests.
- Reproducible GitHub plugin bundle alongside the Python backend distribution.

### Changed

- The historical `premium` first-party atlas is explicitly MIT-licensed and available to all users.
- Python packaging now represents the deterministic backend; the complete Agent plugin is distributed
as a separate GitHub Release bundle.
- Release automation verifies version alignment, tests, package metadata, and installed-wheel behavior
before creating the GitHub Release and publishing to PyPI through Trusted Publishing.

### Validation boundary

- This is a technical preview release of an auditable annotation-review plugin.
- It does not establish biological superiority over CellTypist, SingleR, Azimuth, popV, or expert review.
- A qualified human remains responsible for final annotations and biological claims.

[0.3.0]: https://github.com/HERRY423/CellTypePilot/releases/tag/v0.3.0
68 changes: 44 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
> **Local-first single-cell annotation review plugin for your existing coding workspace.**
> For individual researchers and small labs — no standalone app, no heavy infrastructure.

CellTypePilot uses two coordinated distributions: PyPI provides the deterministic Python
backend, while each GitHub Release provides the complete Agent plugin bundle for Codex and
Claude Code. Installing the backend alone does not install the host plugin manifests or skills.

**CellTypePilot** is a **plugin** for Claude Code / OpenAI Codex. It turns pre-clustered
single-cell data into auditable draft cell-type annotations — with a governed context interface,
**dual-engine** identity scoring (marker overlap + reference embedding), an independent cell-state
Expand Down Expand Up @@ -50,7 +54,7 @@ painful — not because the algorithms don't exist, but because:
| Pain point | What CellTypePilot does |
|---|---|
| **No one reviews your annotations** | A rules-based Annotation Critic checks evidence sufficiency, negative marker conflicts, doublet signals, and ontology consistency — *before* you trust a label |
| **Config barrier is too high** (MCP, pixi, conda...) | `git clone` + `pip install -e .` + run. Zero MCP required for the basic path. `doctor` tells you what you have *before* anything fails |
| **Config barrier is too high** (MCP, pixi, conda...) | Plugin bundle + `pip install celltypepilot` + run. Zero MCP required for the basic path. `doctor` tells you what you have *before* anything fails |
| **Workflow fragmentation** (scripts here, tool there) | Runs inside your Claude Code / Codex session — no context switch to a separate app |
| **Can't explain *why* a cluster got its label** | Every annotation ships with: supporting markers, expression stats, critic flags, and a draft methods paragraph |
| **Rare / transitional states forced into a label** | Low or conflicting evidence produces an explicit `Unknown`/`abstain`; the best candidate is retained separately for review |
Expand Down Expand Up @@ -93,8 +97,8 @@ Web Review also writes governance artifacts when manual overrides are used:
derived evidence/report/figure/manifest artifacts are stale after overrides.

```bash
# 1. Install
pip install -e .
# 1. Install the deterministic backend
pip install celltypepilot

# 2. Environment check — tells you what works and what's missing
celltypepilot doctor
Expand Down Expand Up @@ -257,30 +261,40 @@ separate annotation application or autonomous analysis service.
All three modes call the same Python backend. The `skills/` directory is shared between
both platforms — each reads `SKILL.md` for workflow orchestration.

### Installation
### Distribution and installation

CellTypePilot deliberately separates runtime packaging from Agent-host packaging:

| Distribution | Contains | Intended use |
|---|---|---|
| **PyPI `celltypepilot`** | Python backend, CLI/MCP entry points, bundled MIT atlases, templates | Reproducible runtime installation and upgrades |
| **GitHub plugin bundle** | Codex/Claude manifests, skills, commands, rules, hooks, MCP config, and installable backend source | Complete Agent plugin installation |

The release asset is named `celltypepilot-plugin-<version>.zip` and includes a hashed
`BUNDLE_MANIFEST.json`. Its version must match the PyPI backend version. The repository source
archive is for development; it is not the curated plugin bundle.

```bash
# Claude Code — install as a plugin
# Install the backend from PyPI
pip install celltypepilot
celltypepilot doctor

# Development/source checkout
git clone https://github.com/HERRY423/CellTypePilot ~/.claude/plugins/marketplaces/local/plugins/celltypepilot
cd ~/.claude/plugins/marketplaces/local/plugins/celltypepilot
pip install -e .
# Claude Code discovers .claude-plugin/plugin.json → skills/ + commands/ + hooks/

# Codex — install as a plugin
git clone https://github.com/HERRY423/CellTypePilot ~/.codex/plugins/cache/local/celltypepilot
cd ~/.codex/plugins/cache/local/celltypepilot
pip install -e .
# Codex discovers .codex-plugin/plugin.json → skills/ + agents/openai.yaml

# Standalone CLI — no agent needed
git clone https://github.com/HERRY423/CellTypePilot && cd CellTypePilot
pip install -e .
celltypepilot doctor
```

Optional extras: `pip install -e ".[web]"` (Web Inspector), `"[mcp]"` (native
CellTypePilot MCP server), `"[seurat]"` (.rds support), `"[reference]"` (CellTypist),
`"[embedding]"` (scVI/scANVI), `"[all]"` (everything).
For a release installation, download and extract `celltypepilot-plugin-<version>.zip` from
[GitHub Releases](https://github.com/HERRY423/CellTypePilot/releases), run
`python -m pip install .` inside the extracted directory, then register that directory with
the Agent host. Codex discovers `.codex-plugin/plugin.json`; Claude Code discovers
`.claude-plugin/plugin.json`. Both use the bundled `skills/` and the same Python backend.

Optional backend extras: `pip install "celltypepilot[web]"` (Web Inspector),
`"celltypepilot[mcp]"` (native MCP server), `"celltypepilot[seurat]"` (.rds support),
`"celltypepilot[reference]"` (CellTypist), `"celltypepilot[embedding]"` (scVI/scANVI),
or `"celltypepilot[all]"`.

### Agent-native MCP

Expand Down Expand Up @@ -340,7 +354,7 @@ The CI badge reports the current default-branch state; it is not biological vali

## Built-in Marker Knowledge Graph

The Marker Knowledge Graph (`mkg-2026.08`) covers 80+ cell types across 11 tissues,
The Marker Knowledge Graph (`mkg-2026.08.1`) covers 80+ cell types across 11 tissues,
with positive/negative markers, Cell Ontology IDs, and synonyms. Human and mouse are
both supported with automatic gene symbol conversion.

Expand Down Expand Up @@ -499,8 +513,8 @@ and marks the critic result `AGGREGATE_PROVENANCE_ONLY`.

- [x] **Phase 1 (MVP)** — h5ad adapter, marker knowledge graph, Wilcoxon DE scoring, Annotation Critic, doctor, figures, JSON output, HTML report, methodology draft, manifest provenance, literature validation (PubMed)
- [x] **Phase 2** — Dual-platform plugin packaging (Claude Code `.claude-plugin/` + Codex `.codex-plugin/`), commands, hooks, rules, and optional literature integration
- [x] **Phase 3** — Web Inspector (Flask interactive panel), Seurat .rds adapter, tiered license system (free/academic/commercial), premium atlas (tumor/brain/immune)
- [x] **Phase 4** — Reference Embedding + Ensemble fusion (CellTypist / scANVI / KNN / Correlation backends), adaptive weighting, transitional state detection, ensemble-aware critic, RSA-2048 license security, sparse-preserving Seurat conversion, Web Inspector override API
- [x] **Phase 3** — Web Inspector (Flask interactive panel), Seurat .rds adapter, and extended first-party atlas (tumor/brain/immune)
- [x] **Phase 4** — Reference Embedding + Ensemble fusion (CellTypist / scANVI / KNN / Correlation backends), adaptive weighting, transitional state detection, ensemble-aware critic, sparse-preserving Seurat conversion, Web Inspector override API
- [x] **Architecture hardening** — Orchestrator layer (pipeline logic extracted from CLI), Jinja2 templates, multi-species detection, synonym-based tissue detection, and Python 3.10–3.12 CI
- [x] **Phase 5** — Governed Context Pack, custom marker trust boundary, legal identity ontology IDs, and independent Identity × State outputs
- [ ] **Validation release** — The immutable public registry, donor-aware release builder,
Expand All @@ -511,4 +525,10 @@ and marks the critic result `AGGREGATE_PROVENANCE_ONLY`.

## License

MIT
CellTypePilot source code, the core atlas, and the historical `premium` first-party atlas are
released under the [MIT License](LICENSE). The `premium` directory name is retained for
backward compatibility; it no longer denotes a paid or license-gated content tier.

Third-party references, imported datasets, and community extension packs retain their own
licenses and provenance. An installed pack's license metadata does not convert that content to
MIT. Biological outputs remain reviewable drafts requiring qualified human adjudication.
Loading