Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
225 changes: 26 additions & 199 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,224 +1,51 @@
# md-docrs-proxy
# md-docrs

`md-docrs-proxy` resolves docs.rs rustdoc JSON URLs and renders rustdoc JSON as Markdown.
`md-docrs` is a CLI that renders docs.rs rustdoc JSON as Markdown.

## Workspace

Rust crates under `crates/`:

- `md-docrs-core` — shared spec parsing, docs.rs resolution, rustdoc JSON rendering, cache traits
- `md-docrs-fetch-http` — native HTTP fetcher for docs.rs
- `md-docrs-cli` — native CLI that prints Markdown to stdout
- `md-docrs-server` — native HTTP server
- `md-docrs-worker` — Cloudflare Worker crate
- `md-docrs-rust-wasm` — Rust `wasm32-unknown-unknown` export layer
- `md-docrs-wasm-compare` — host-side WASM comparison harness

Other top-level directories:

- `zig/` — Zig implementation of the minimal `resolve_url` ABI, plus its Worker wrapper
- `wasm/` — staged WASM artifacts and the repo-level build script

## What each path owns

- `crates/` owns the Rust implementation
- `zig/` owns the minimal Zig implementation
- `wasm/` owns artifact staging for Rust/Zig WASM comparison

The top-level `wasm/` directory is not a Cargo crate.

## Build and test

Build the Rust workspace:

```sh
cargo build --workspace
```

Run the Rust tests:

```sh
cargo test --workspace
```

Run the Zig tests from the repo root:

```sh
zig build test --build-file zig/lib/build.zig
```

## Release packaging

This repository is configured for [`cargo-dist`](https://axodotdev.github.io/cargo-dist/)
releases of the `md-docrs` CLI. With `cargo-dist` 0.31, the generated release
configuration lives in [dist-workspace.toml](/Users/thomas/dev/perso/github/md_docrs_proxy/dist-workspace.toml)
and the CI workflow lives in [.github/workflows/release.yml](/Users/thomas/dev/perso/github/md_docrs_proxy/.github/workflows/release.yml).

Install the release tooling locally:

```sh
cargo install cargo-dist --locked
cargo install cargo-release --locked
```

Validate what the release workflow will build:

```sh
dist plan --tag vX.Y.Z
```

Build the current platform's release artifacts and installers locally:

```sh
dist build --tag vX.Y.Z
```

Release tags should use the unified workspace form (`v0.1.0`, `v0.2.3`, ...).
This workspace distributes only the `md-docrs-cli` package with `cargo-dist`,
which ships the `md-docrs` binary.

Homebrew releases are published through the `ThomAub/homebrew-tap` tap managed
by `cargo-dist`. There is intentionally no checked-in `HomebrewFormula/`
directory in this repository.

This repository configures `cargo-release` to:

- update the shared Rust workspace version in one place
- create a single unified release tag as `v{{version}}`
- publish the crates needed by the CLI release
- keep the final remote push explicit

For this virtual workspace, preview and execute releases from the repository root:

```sh
# preview
cargo release 0.2.0

# version bump + release commit + tag + crates.io publish, but keep the push explicit
cargo release 0.2.0 --execute --no-push

# push the release commit and tag after validation
git push origin main
git push origin v0.2.0
```

Repository prerequisites:

- Create the tap repository `ThomAub/homebrew-tap`.
- Add a GitHub personal access token with `repo` scope as the
`HOMEBREW_TAP_TOKEN` secret in this repository.

Once a release has been published, install with Homebrew:

```sh
brew install ThomAub/tap/md-docrs
```

## Native CLI

The CLI binary comes from `md-docrs-cli`.

Spec grammar:
It lets you query a crate, module, type, trait, function, or other documented item with a compact spec:

```text
crate[@version][::path::to::item]
```

Examples:
## Quick start

Run the CLI directly from the workspace:

```sh
```bash
cargo run -p md-docrs-cli -- anyhow
cargo run -p md-docrs-cli -- anyhow::Error
cargo run -p md-docrs-cli -- tokio::sync::Mutex
cargo run -p md-docrs-cli -- tokio@1.52.1::sync::Mutex
cargo run -p md-docrs-cli -- --target x86_64-unknown-linux-gnu tokio::sync::Mutex
```

Output is Markdown on stdout.
Output is written to stdout as Markdown.

## Native server
## What this repository contains

The HTTP server binary comes from `md-docrs-server`.
This repository is a Rust workspace centered on the `md-docrs` CLI, plus related server, worker, and WASM targets for some explorative work on wasm and zig.

Start it locally:
### Crates

```sh
cargo run -p md-docrs-server -- --port 8080 --bind 127.0.0.1
```

Example requests:

```sh
curl -sS http://127.0.0.1:8080/anyhow
curl -sS http://127.0.0.1:8080/anyhow/latest/anyhow/struct.Error.html
curl -sS http://127.0.0.1:8080/tokio/latest/tokio/sync/struct.Mutex.html
curl -sS http://127.0.0.1:8080/healthz
```

Response behavior:
- `crates/md-docrs-cli` — the `md-docrs` command-line tool
- `crates/md-docrs-core` — spec parsing, docs.rs resolution, rendering, and cache abstractions
- `crates/md-docrs-fetch-http` — native HTTP fetcher for docs.rs rustdoc JSON
- `crates/md-docrs-server` — native HTTP server that serves Markdown
- `crates/md-docrs-worker` — Cloudflare Worker for a hosted HTTP version
- `crates/md-docrs-rust-wasm` — Rust WASM export layer
- `crates/md-docrs-wasm-compare` — host-side comparison harness for WASM builds

- `200` with `Content-Type: text/markdown; charset=utf-8`
- `400` for invalid specs
- `404` for missing items
- `502` for upstream, decode, or JSON errors
### Other directories

Optional disk-backed cache support is available behind the `hybrid-cache` feature on `md-docrs-server`.
- `zig/` — minimal Zig URL-resolution implementation for fun and compare WASM size
- `wasm/` — WASM artifacts and comparison build flow

## Rust WASM

The Rust WASM crate lives at `crates/md-docrs-rust-wasm`.

Minimal build, ABI-compatible with the Zig module:

```sh
cargo build --profile wasm-release --target wasm32-unknown-unknown \
-p md-docrs-rust-wasm --no-default-features
```
## Hosted MCP

Default build adds `render_markdown`:
There is also a Cloudflare Worker-based hosted path in `crates/md-docrs-worker`.
Ongoing work!

```sh
cargo build --profile wasm-release --target wasm32-unknown-unknown \
-p md-docrs-rust-wasm
```

Full build adds `render_markdown` and `render_spec`:

```sh
cargo build --profile wasm-release --target wasm32-unknown-unknown \
-p md-docrs-rust-wasm --no-default-features --features full
```

## Zig

The Zig subtree implements the minimal `resolve_url` path.

Common commands:

```sh
zig build --build-file zig/lib/build.zig
zig build cli --build-file zig/lib/build.zig
zig build test --build-file zig/lib/build.zig
```

See `zig/README.md` for details.

## WASM comparison

Stage artifacts, then run the comparison harness:

```sh
./wasm/build.sh
cargo run -p md-docrs-wasm-compare -- --offline
```

See `wasm/README.md` for the workflow and supported flags.

## Notes

Current limits:
## Release packaging

- in-memory cache by default for native paths
- no disk cache unless `md-docrs-server` is built with `hybrid-cache`
- partial rendering coverage; not all rustdoc surfaces are rendered yet
- Zig currently covers URL resolution only, not fetch/decompress/render
This repository is configured thanks to [`cargo-dist`](https://axodotdev.github.io/cargo-dist/) and [`cargo-release`](https://github.com/crate-ci/cargo-release)
93 changes: 93 additions & 0 deletions crates/md-docrs-cli/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# md-docrs-cli

`md-docrs-cli` provides the `md-docrs` command-line interface for rendering docs.rs Rust API documentation as Markdown.

It resolves a crate or item spec, fetches the corresponding rustdoc JSON from docs.rs, and prints Markdown to stdout.

## Install

From the workspace root:

```bash
cargo install --path crates/md-docrs-cli
```

Run without installing:

```bash
cargo run -p md-docrs-cli -- <SPEC>
```

## Spec format

```text
crate[@version][::path::to::item]
```

Examples:

```bash
md-docrs anyhow
md-docrs anyhow::Error
md-docrs tokio::sync::Mutex
md-docrs tokio@1.52.1::sync::Mutex
md-docrs --target x86_64-unknown-linux-gnu tokio::sync::Mutex
```

## Usage

```text
md-docrs [--target <TRIPLE>] <SPEC>
```

Arguments:

- `<SPEC>` — crate or item to render
- `--target <TRIPLE>` — optional target triple to resolve a target-specific rustdoc JSON build

## Examples

Render a crate landing page as Markdown:

```bash
cargo run -p md-docrs-cli -- anyhow
```

Render a specific item:

```bash
cargo run -p md-docrs-cli -- anyhow::Error
```

Render a version-pinned item:

```bash
cargo run -p md-docrs-cli -- tokio@1.52.1::sync::Mutex
```

Render using a specific target triple:

```bash
cargo run -p md-docrs-cli -- --target x86_64-unknown-linux-gnu tokio::sync::Mutex
```

Pipe the Markdown into a file:

```bash
cargo run -p md-docrs-cli -- serde::Serialize > serde-serialize.md
```

## Output

The command writes Markdown to standard output.

On invalid input or upstream failures, it exits with an error and prints a message to standard error.

## Related crates

- `md-docrs-core` — spec parsing, resolution, rendering, and cache abstractions
- `md-docrs-fetch-http` — native HTTP fetcher used by the CLI

## Workspace

This crate lives in the `md-docrs-proxy` workspace under `crates/md-docrs-cli`.
Loading
Loading