Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ location: ~/m-dev-tools/m-test-engine
exposes:
container_name: "m-test-engine"
image_base: "yottadb/yottadb-base:latest-master"
mount_point: "/work"
mount_point: "/m-work"
lifecycle: "make up | down | smoke | shell | clean | logs"

consumes:
Expand All @@ -38,7 +38,7 @@ docs:

Minimal YottaDB Docker container for `m-cli` and `m-stdlib` testing.
Long-running container exposing a YottaDB engine via `docker exec`.
Consumer projects bind-mount their source as `/work` and dispatch
Consumer projects coordinate through a shared `/m-work` bind mount and dispatch
`docker exec m-test-engine $ydb_dist/mumps -run ...` commands.

The full design rationale and lifecycle table is in `README.md`.
Expand All @@ -49,7 +49,7 @@ The full design rationale and lifecycle table is in `README.md`.
`compose.yml` plus thin Makefile wrappers (`up`, `down`, `smoke`,
`shell`, `clean`, `logs`).
- A container name (`m-test-engine`) and a bind-mount contract
(`/work` in container = consumer's `$PWD` on host).
(`/m-work` in container = host's `/m-work`, the shared m-* working tree).
- A `dist/lifecycle.json` describing those facts in
machine-readable form for the org-level AI-discoverability catalog.

Expand Down Expand Up @@ -125,7 +125,7 @@ make check-docs-prose # docs/ holds only prose (this repo has no docs/ at all)
the date only when the manifest changes materially (image bump,
container rename, mount-point change).
- **Container name and mount point are public contract.**
`m-test-engine` (name) and `/work` (mount) are referenced by
`m-test-engine` (name) and `/m-work` (mount) are referenced by
consumer code paths in m-cli and m-stdlib. Renaming either is a
breaking change requiring coordinated updates in those repos.
- **Image base is pinned for a reason.** The Dockerfile pin
Expand Down
37 changes: 30 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,11 @@ YottaDB + VistA + RPC broker + FileMan + Octo SQL stack. For non-VistA
M development, that's overkill: the tests just need a YottaDB engine.

`m-test-engine` is exactly that — `yottadb/yottadb-base:latest-master`
with a keep-alive command. No SSH server, no VistA, no extras. The
consumer project's source bind-mounts in via `/work`; routines are
compiled and run from there with no SCP / staging round-trip.
with a keep-alive command. No SSH server, no VistA, no extras. All
participating m-* repos live under `/m-work` on the host (clones or
symlinks) and the container bind-mounts that single directory as
`/m-work`, so every repo's routines are simultaneously visible inside
one running engine — no per-cwd remount, no SCP / staging round-trip.

For VistA-specific work, keep using vista-meta. This repo is the
lightweight default for the rest of the toolchain.
Expand Down Expand Up @@ -69,18 +71,39 @@ docker exec m-test-engine bash -lc \
```

The `<stage>` is the in-container path to the consumer project's
routine dirs — derived by mapping the project root through the
`/work` bind mount. There's no SCP / SFTP step: the consumer's
routine dirs — derived by mapping the project root through the shared
`/m-work` bind mount. There's no SCP / SFTP step: every m-* repo's
filesystem is directly visible inside the container.

`m-stdlib`'s test runner uses the same transport once Track A3 of the
[m-dev-tools-todo plan](../m-dev-tools-todo.md) lands.

## Machine-readable contract

[`dist/m-test-engine.json`](dist/m-test-engine.json) is the canonical
machine-readable contract between this repo and its consumers
(primarily `m-cli`'s `m doctor` and the upcoming `m engine` subcommand
family). It declares the image registry, default tag, container name,
bind-mount layout, compose file path, minimum Docker version, YottaDB
release, and a `run_args` block sufficient to reconstruct an equivalent
`docker run` invocation when the compose plugin is unavailable.

The schema lives alongside it at
[`dist/m-test-engine.schema.json`](dist/m-test-engine.schema.json).
`make check-manifest` validates both `dist/repo.meta.json` and
`dist/m-test-engine.json` — schema-shape, `compose_file` resolves on
disk, and `verified_on` is within 90 days.

Consumers vendor `dist/m-test-engine.json` at release time (m-cli pins
to a tagged release of this repo). The protocol bump policy and the
broader integration design are recorded in
[`docs/m-engine-implementation-plan.md`](docs/m-engine-implementation-plan.md).

## Configuration

| Env var | Default | Purpose |
|-----------------------|---------|-----------------------------------------------------------|
| `M_TEST_ENGINE_BIND` | `$PWD` | Override the host directory mounted as `/work`. |
| `M_TEST_ENGINE_BIND` | `/m-work` | Override the host directory mounted as `/m-work`. |

## Optional: SSH overlay

Expand All @@ -95,7 +118,7 @@ PR if you need it.
| | m-test-engine | vista-meta |
|---|---|---|
| YottaDB | ✓ | ✓ |
| Bind-mount source code | ✓ (via `/work`) | ✓ (via `/home/vehu/dev/r`) |
| Bind-mount source code | ✓ (via `/m-work`) | ✓ (via `/home/vehu/dev/r`) |
| VistA + FileMan | — | ✓ |
| RPC broker / VistALink | — | ✓ |
| Octo SQL / YDB GUI | — | ✓ |
Expand Down
4 changes: 2 additions & 2 deletions dist/lifecycle.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
"hostname": "m-test-engine",
"image": "m-test-engine:latest",
"image_base": "yottadb/yottadb-base:latest-master",
"mount_point": "/work",
"mount_point": "/m-work",
"mount_source_env": "M_TEST_ENGINE_BIND",
"mount_source_default": "$PWD",
"mount_source_default": "/m-work",
"globals_volume": "m-test-engine-globals",
"ports": [],
"ssh_server": false,
Expand Down
33 changes: 33 additions & 0 deletions dist/m-test-engine.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"$schema": "https://raw.githubusercontent.com/m-dev-tools/m-test-engine/main/dist/m-test-engine.schema.json",
"protocol": 1,
"image": "ghcr.io/m-dev-tools/m-test-engine",
"default_tag": "r2.02",
"container": "m-test-engine",
"bind_mount": {
"host": "/m-work",
"container": "/m-work",
"mode": "rw"
},
"compose_file": "docker/compose.yml",
"repo_url": "https://github.com/m-dev-tools/m-test-engine",
"min_docker": "20.10",
"ydb_version": "r2.02",
"run_args": {
"hostname": "m-test-engine",
"working_dir": "/m-work",
"restart": "unless-stopped",
"volumes": [
{
"name": "m-test-engine-globals",
"target": "/data"
}
],
"command": [
"tail",
"-f",
"/dev/null"
]
},
"verified_on": "2026-05-11"
}
130 changes: 130 additions & 0 deletions dist/m-test-engine.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/m-dev-tools/m-test-engine/main/dist/m-test-engine.schema.json",
"title": "m-test-engine manifest",
"description": "Public machine-readable contract for the m-test-engine YottaDB Docker container. Consumed by m-cli's `m doctor` and the `m engine` subcommand family. See docs/m-engine-implementation-plan.md for design rationale and the protocol bump policy.",
"type": "object",
"additionalProperties": false,
"required": [
"protocol",
"image",
"default_tag",
"container",
"bind_mount",
"compose_file",
"repo_url",
"min_docker",
"ydb_version",
"run_args",
"verified_on"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri"
},
"protocol": {
"type": "integer",
"minimum": 1,
"description": "Contract version. Bumped when a field is removed, renamed, or has its semantics changed. Additive evolution (new optional fields) does NOT bump."
},
"image": {
"type": "string",
"description": "Canonical image reference without tag (e.g. ghcr.io/m-dev-tools/m-test-engine). Combined with default_tag for the full reference."
},
"default_tag": {
"type": "string",
"description": "Default tag to pull when no override is given (e.g. r2.02)."
},
"container": {
"type": "string",
"description": "Container name. Public contract — m-cli's DockerEngine, m-stdlib's test runner, and every `docker exec` call reference this exact name."
},
"bind_mount": {
"type": "object",
"additionalProperties": false,
"required": ["host", "container", "mode"],
"description": "Single shared bind mount. Host directory contains all participating m-* repo checkouts; container sees identical layout.",
"properties": {
"host": {
"type": "string",
"description": "Absolute host filesystem path."
},
"container": {
"type": "string",
"description": "Absolute in-container path of the bind mount."
},
"mode": {
"type": "string",
"enum": ["ro", "rw"],
"description": "Mount mode."
}
}
},
"compose_file": {
"type": "string",
"description": "Repo-relative path to the canonical compose.yml. Primary path for `m engine start`."
},
"repo_url": {
"type": "string",
"format": "uri",
"description": "Repo URL for issue links and source-of-truth pointers."
},
"min_docker": {
"type": "string",
"pattern": "^\\d+\\.\\d+(\\.\\d+)?$",
"description": "Minimum Docker Engine version required (e.g. 20.10)."
},
"ydb_version": {
"type": "string",
"description": "YottaDB release shipped in the image (e.g. r2.02). Sourced from the image base."
},
"run_args": {
"type": "object",
"additionalProperties": false,
"required": ["hostname", "working_dir", "restart", "volumes", "command"],
"description": "Fallback args for `docker run` when the compose plugin is unavailable. Consumer reconstructs an equivalent invocation from these fields + image + bind_mount.",
"properties": {
"hostname": {
"type": "string"
},
"working_dir": {
"type": "string"
},
"restart": {
"type": "string",
"enum": ["no", "always", "on-failure", "unless-stopped"]
},
"volumes": {
"type": "array",
"description": "Named Docker volumes beyond the primary bind_mount.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "target"],
"properties": {
"name": {
"type": "string",
"description": "Named Docker volume identifier."
},
"target": {
"type": "string",
"description": "Absolute in-container mount target."
}
}
}
},
"command": {
"type": "array",
"items": { "type": "string" },
"description": "Container entrypoint command (overrides Dockerfile CMD if set)."
}
}
},
"verified_on": {
"type": "string",
"format": "date",
"description": "ISO date the manifest was last verified against the live image. Stale manifests >90 days fail check-manifest."
}
}
}
11 changes: 6 additions & 5 deletions dist/repo.meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@
"language": ["dockerfile"],
"license": "AGPL-3.0",
"agent_instructions": "AGENTS.md",
"verified_on": "2026-05-10",
"verified_on": "2026-05-11",
"exposes": {
"lifecycle": "dist/lifecycle.json",
"dockerfile": "docker/Dockerfile",
"compose": "docker/compose.yml"
"lifecycle": "dist/lifecycle.json",
"engine_contract": "dist/m-test-engine.json",
"dockerfile": "docker/Dockerfile",
"compose": "docker/compose.yml"
},
"verification_commands": ["make smoke", "make check-manifest"],
"status": "active",
"notes": "Container name 'm-test-engine' and mount point '/work' are public contract — m-cli's DockerEngine and m-stdlib's test runner reference them. Image base 'yottadb-base:latest-master' is pinned to match m-stdlib's CI."
"notes": "Container name 'm-test-engine' and mount point '/m-work' are public contract — m-cli's DockerEngine and m-stdlib's test runner reference them via `dist/m-test-engine.json`. Image base 'yottadb-base:latest-master' is pinned to match m-stdlib's CI."
}
15 changes: 8 additions & 7 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
#
# docker exec m-test-engine bash -lc '$ydb_dist/mumps -run ^SUITE'
#
# The companion compose.yml bind-mounts the consumer project's $PWD as
# /work, so a project's routine dirs (src/, routines/, tests/, etc.)
# become /work/src, /work/routines, /work/tests inside the container.
# m-cli's DockerEngine.stage_routines() generates the right
# `ydb_routines` value for those in-container paths.
# The companion compose.yml bind-mounts the host's /m-work directory
# (containing all m-* repo checkouts) as /m-work inside the container.
# A consumer project at /m-work/m-cli/ on the host becomes
# /m-work/m-cli/ inside the container; m-cli's DockerEngine assembles
# `ydb_routines` from the appropriate subdirs without per-cwd remounts.
#
# Pinned to yottadb-base:latest-master to match what m-stdlib's CI uses.

Expand All @@ -26,8 +26,9 @@ RUN echo '. /opt/yottadb/current/ydb_env_set 2>/dev/null || true' \
&& chmod +x /etc/profile.d/ydb-env.sh

# Working directory matches the bind-mount target. The compose file's
# bind makes /work == the consumer's $PWD on the host.
WORKDIR /work
# bind makes /m-work == the host's /m-work directory (the shared
# m-* working tree).
WORKDIR /m-work

# The container needs to stay running so `docker exec` calls can target
# it. tail -f /dev/null is the canonical "do nothing forever" pattern;
Expand Down
25 changes: 15 additions & 10 deletions docker/compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@
#
# docker compose -f $(M_TEST_ENGINE)/docker/compose.yml up -d
#
# (or use the consumer project's `make engine-up` target, which
# delegates here once Track A3 lands).
# Bind mount: /m-work in the container = the host's /m-work directory.
# The host /m-work is the shared working tree containing all m-* repo
# checkouts (m-cli, m-stdlib, m-test-engine, m-modern-corpus, ...) so a
# single running engine exposes routines from every participating repo
# without per-cwd container churn. Override with M_TEST_ENGINE_BIND
# (host path) if you keep your m-* repos elsewhere.
#
# Bind mount: /work in the container = the caller's $PWD on the host.
# Override with the M_TEST_ENGINE_BIND env var to mount somewhere else.
# See `dist/m-test-engine.json` for the full machine-readable contract.

name: m-test-engine

Expand All @@ -20,17 +23,19 @@ services:
image: m-test-engine:latest
container_name: m-test-engine
hostname: m-test-engine
restart: unless-stopped

volumes:
# Bind the caller's working directory (or M_TEST_ENGINE_BIND
# override) as /work inside the container.
- ${M_TEST_ENGINE_BIND:-${PWD}}:/work
# Shared host bind: /m-work → /m-work. All m-* repos live under
# /m-work on the host (clones or symlinks). Override host side
# with M_TEST_ENGINE_BIND if your layout differs.
- ${M_TEST_ENGINE_BIND:-/m-work}:/m-work
# Persistent globals so test data survives container restarts.
# m-cli/m-stdlib tests typically clear globals per-test, but the
# volume protects against accidental loss between sessions.
# Tests typically clear globals per-test, but the volume protects
# against accidental loss between sessions.
- m-test-engine-globals:/data

working_dir: /work
working_dir: /m-work

volumes:
m-test-engine-globals:
Expand Down
22 changes: 22 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
created: 2026-05-11
last_modified: 2026-05-11
revisions: 0
doc_type: [REFERENCE]
---

# m-test-engine — Documentation Index

> First-pass index generated 2026-05-11. Labels follow the shared vocabulary below; the same vocabulary is used across all m-dev-tools repos.

## Vocabulary

Each doc is labeled `[TYPE · type? · connection · connection?]`.

**Types** — `HISTORY` · `ARCHITECTURE` · `DESIGN` · `ADR` · `SPEC` · `REFERENCE` · `GUIDE` · `TUTORIAL` · `ROADMAP` · `PLAN` · `RESEARCH` · `SURVEY` · `GAP-ANALYSIS` · `STATUS` · `EXPLAINER` · `NOTES` · `WORKED-EXAMPLE` · `SETUP` · `INTEGRATION` · `PROPOSAL` · `BUILD-LOG` · `CHANGELOG` · `POSTMORTEM`

**Repo connections** — `history` · `function` · `design` · `architecture` · `planning` · `implementation`

## Top-level

- **`m-cli-integration-research.md`** — `[RESEARCH · DESIGN · architecture · planning]` Pre-design synthesis tracing the m-cli/m-test-engine contract, discovery layers, manifest sketch, and `m engine` subcommand family.
Loading
Loading