From 7336614740fff8cd54fb0763af1544da086df7e9 Mon Sep 17 00:00:00 2001 From: ExplodingKonjac Date: Fri, 29 May 2026 22:26:08 +0800 Subject: [PATCH 1/3] fix: macro compression issue --- .claude/project-tracker/.meta | 37 ---- .claude/project-tracker/INDEX.md | 49 ----- .claude/project-tracker/api.md | 72 ------- .claude/project-tracker/architecture.md | 72 ------- .claude/project-tracker/data-model.md | 88 -------- .claude/project-tracker/deployment.md | 59 ----- .claude/project-tracker/implementation.md | 79 ------- .claude/project-tracker/modules/cli.md | 47 ---- .claude/project-tracker/modules/core.md | 39 ---- .../modules/vscode-extension.md | 88 -------- .claude/project-tracker/progress.md | 47 ---- .claude/project-tracker/stack.md | 49 ----- .claude/project-tracker/toolchain.md | 81 ------- .project-tracker/.state.json | 204 ++++++++++++++++++ .project-tracker/INDEX.md | 74 +++++++ .project-tracker/api.md | 56 +++++ .project-tracker/architecture.md | 69 ++++++ .project-tracker/conventions.md | 89 ++++++++ .project-tracker/data-model.md | 44 ++++ .project-tracker/deployment.md | 54 +++++ .project-tracker/implementation.md | 56 +++++ .project-tracker/modules/cli.md | 33 +++ .project-tracker/modules/core.md | 34 +++ .project-tracker/modules/vscode.md | 38 ++++ .project-tracker/progress.md | 44 ++++ .project-tracker/stack.md | 51 +++++ .project-tracker/toolchain.md | 58 +++++ Cargo.lock | 6 +- texpand-cli/Cargo.toml | 2 +- texpand-core/Cargo.toml | 2 +- texpand-core/src/compressor.rs | 44 ++-- texpand-core/src/expander.rs | 34 ++- texpand-vscode/Cargo.toml | 2 +- texpand-vscode/extension/package.json | 2 +- 34 files changed, 969 insertions(+), 834 deletions(-) delete mode 100644 .claude/project-tracker/.meta delete mode 100644 .claude/project-tracker/INDEX.md delete mode 100644 .claude/project-tracker/api.md delete mode 100644 .claude/project-tracker/architecture.md delete mode 100644 .claude/project-tracker/data-model.md delete mode 100644 .claude/project-tracker/deployment.md delete mode 100644 .claude/project-tracker/implementation.md delete mode 100644 .claude/project-tracker/modules/cli.md delete mode 100644 .claude/project-tracker/modules/core.md delete mode 100644 .claude/project-tracker/modules/vscode-extension.md delete mode 100644 .claude/project-tracker/progress.md delete mode 100644 .claude/project-tracker/stack.md delete mode 100644 .claude/project-tracker/toolchain.md create mode 100644 .project-tracker/.state.json create mode 100644 .project-tracker/INDEX.md create mode 100644 .project-tracker/api.md create mode 100644 .project-tracker/architecture.md create mode 100644 .project-tracker/conventions.md create mode 100644 .project-tracker/data-model.md create mode 100644 .project-tracker/deployment.md create mode 100644 .project-tracker/implementation.md create mode 100644 .project-tracker/modules/cli.md create mode 100644 .project-tracker/modules/core.md create mode 100644 .project-tracker/modules/vscode.md create mode 100644 .project-tracker/progress.md create mode 100644 .project-tracker/stack.md create mode 100644 .project-tracker/toolchain.md diff --git a/.claude/project-tracker/.meta b/.claude/project-tracker/.meta deleted file mode 100644 index 29a3580..0000000 --- a/.claude/project-tracker/.meta +++ /dev/null @@ -1,37 +0,0 @@ -files: - INDEX.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - stack.md: - baseline: a3a0e2c9e1c1f5f0a3a0e2c9e1c1f5f0a3a0e2c9 - updated: 2026-05-14T16:00:00Z - toolchain.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - architecture.md: - baseline: a3a0e2c9e1c1f5f0a3a0e2c9e1c1f5f0a3a0e2c9 - updated: 2026-05-14T16:00:00Z - progress.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - implementation.md: - baseline: a3a0e2c9e1c1f5f0a3a0e2c9e1c1f5f0a3a0e2c9 - updated: 2026-05-14T16:00:00Z - data-model.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - api.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - deployment.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - modules/core.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - modules/cli.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z - modules/vscode-extension.md: - baseline: 65b49e172b12d9f2eda018c1f2e146eaacaf9c10 - updated: 2026-05-12T00:42:00Z diff --git a/.claude/project-tracker/INDEX.md b/.claude/project-tracker/INDEX.md deleted file mode 100644 index 6ce76d4..0000000 --- a/.claude/project-tracker/INDEX.md +++ /dev/null @@ -1,49 +0,0 @@ -# Template-Expand - -C/C++ `#include` template expansion tool for Competitive Programming. Expands local headers into a single file, with optional semantic-safe token-level compression. Ships as both a CLI tool (`texpand`) and a VSCode extension. - -## Table of Contents - -- [Index](INDEX.md) — this file -- [Stack](stack.md) — technology choices & rationale -- [Toolchain](toolchain.md) — build, lint, test, CI/CD -- [Architecture](architecture.md) — module layout & data flow -- [Progress](progress.md) — current status & roadmap -- [Implementation](implementation.md) — entry points & key logic -- [Data Model](data-model.md) — core types & state -- [API](api.md) — CLI surface & extension commands -- [Deployment](deployment.md) — building & packaging -- [Modules](modules/) — per-crate deep dives - - [Core](modules/core.md) — `texpand-core` library - - [CLI](modules/cli.md) — `texpand-cli` frontend - - [VSCode Extension](modules/vscode-extension.md) — `texpand-vscode` frontend - -## Tech Stack Summary - -- **Language**: Rust 2024 edition (stable toolchain) -- **Parser**: tree-sitter C/C++ grammar for AST-based include analysis -- **CLI**: clap argument parsing, `~/.config/texpand.toml` config -- **VSCode**: WASM-WASI process mode via `@vscode/wasm-wasi`, TypeScript extension host -- **Serialization**: serde/serde_json for structured output - -## Quick-Reference Commands - -```bash -# Build -cargo build --workspace - -# Test -cargo test --workspace - -# Lint -cargo clippy --workspace -- -D warnings - -# Format -cargo fmt --all - -# Run CLI -cargo run -p texpand-cli -- main.cpp -c - -# Build VSCode WASM -cargo build -p texpand-vscode --target wasm32-wasip1 --release -``` diff --git a/.claude/project-tracker/api.md b/.claude/project-tracker/api.md deleted file mode 100644 index f8174f0..0000000 --- a/.claude/project-tracker/api.md +++ /dev/null @@ -1,72 +0,0 @@ -# API Surface - -## CLI (`texpand`) - -### Command -``` -texpand [OPTIONS] - INPUT: Path to C/C++ source file (use "-" for stdin) -``` - -### Options - -| Flag | Description | -|------|-------------| -| `-c, --compress` | Enable token-level compression (overrides config) | -| `--no-compress` | Disable compression (overrides config) | -| `-i, --include ` | Add include search path (repeatable, overrides config) | -| `-o, --output ` | Write to file instead of stdout | -| `-C, --clipboard` | Copy to clipboard (mutually exclusive with `-o`) | -| `--config ` | Config file path (default: `~/.config/texpand.toml`) | - -### Exit Codes -- 0: Success -- Non-zero: Error (anyhow prints diagnostic to stderr) - -## VSCode Extension - -### Commands -| Command ID | Title | Behavior | -|-----------|-------|----------| -| `texpand.expandDefault` | Texpand: Expand Current File (Default) | Uses configured `outputMode` | -| `texpand.expandAndCopy` | Texpand: Expand and Copy | Forces clipboard output | -| `texpand.expandToNewFile` | Texpand: Expand to New File | Creates `.expanded.cpp` | - -### Activation -Events: `onLanguage:c`, `onLanguage:cpp` - -### WASM Process Protocol -The TypeScript extension: -1. Spawns a WASI process from the WASM binary -2. Sets env vars: `TEXPAND_ENTRY_PATH`, `TEXPAND_COMPRESS`, `TEXPAND_INCLUDE_PATHS` -3. Mounts workspace filesystem paths -4. Reads JSON from WASM stdout: `{ success: bool, data?: string, error?: string }` -5. Displays result (clipboard or new file) - -## Core Library (`texpand-core`) - -### Public API -```rust -// expander.rs -pub fn expand( - entry_path: &Path, - entry_source: &str, - resolver: &dyn FileResolver, - opts: &ExpandOptions, -) -> Result - -pub struct ExpandOptions { - pub compress: bool, -} - -pub trait FileResolver { - fn resolve(&self, includer_path: &Path, include_path: &str) -> Result; - fn read_content(&self, resolved_path: &Path) -> Result; -} - -// Standalone compression -pub fn compress(tree: &Tree, source: &str) -> String; -pub fn compress_stripped(tree: &Tree, source: &str) -> String; -``` - -No REST API, no HTTP endpoints — this is a local-only tool. diff --git a/.claude/project-tracker/architecture.md b/.claude/project-tracker/architecture.md deleted file mode 100644 index 7d6725d..0000000 --- a/.claude/project-tracker/architecture.md +++ /dev/null @@ -1,72 +0,0 @@ -# Architecture - -## Overview - -Monorepo with a shared core library and two frontends. The core is I/O-free — all file reading goes through the `FileResolver` trait, allowing each frontend to provide its own storage backend. - -``` -┌──────────────────────────────────────────────────────┐ -│ texpand-core │ -│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ -│ │ parser │ │ expander │ │ compressor │ │ -│ │ (tree- │ │ (BFS │ │ (token-level AST │ │ -│ │ sitter) │ │ expand) │ │ leaf walk) │ │ -│ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │ -│ │ │ │ │ -│ ┌────┴──────────────┴──────────────────┴─────────┐ │ -│ │ resolver.rs (FileResolver trait) │ │ -│ │ resolve() + read_content() — no std::fs calls │ │ -│ └────────────────────┬────────────────────────────┘ │ -└───────────────────────┼──────────────────────────────┘ - │ - ┌─────────────┼─────────────┐ - │ │ │ -┌─────────┴──────┐ ┌───┴────────┐ ┌──┴────────────┐ -│ texpand-cli │ │ texpand- │ │ texpand- │ -│ FsResolver │ │ vscode │ │ vscode │ -│ (std::fs) │ │ (WASI fs) │ │ (TypeScript) │ -│ │ │ │ │ │ -│ clap CLI │ │ env-var │ │ VSCode │ -│ config/toml │ │ config │ │ commands │ -│ arboard clip │ │ JSON stdout│ │ WASM launcher │ -└────────────────┘ └────────────┘ └───────────────┘ -``` - -## Module Breakdown (texpand-core) - -| Module | File | Responsibility | -|--------|------|---------------| -| `resolver` | `resolver.rs` | `FileResolver` trait — abstract file I/O | -| `parser` | `parser.rs` | tree-sitter C/C++ parse + include extraction | -| `expander` | `expander.rs` | BFS recursive expansion, cycle detection, preproc context tracking | -| `compressor` | `compressor.rs` | Token-level AST leaf walk compression | - -## Key Data Flow - -``` -Source file ──→ parser.rs ──→ AST tree - │ - ▼ - extract_includes() - │ - ▼ - expander.rs ──→ FileResolver.resolve() - │ │ - ▼ ▼ - Recursive expand ──→ read_content() - │ - ▼ - Compressor (optional) ──→ Final string output -``` - -## Security Boundaries - -- **I/O boundary**: `FileResolver` trait is the sole I/O interface — no `std::fs` or `std::io` in `texpand-core` -- **WASM sandbox**: VSCode extension runs in WASI process with virtual filesystem — no access to host filesystem beyond workspace -- **Fork isolation**: Linux clipboard uses `fork()` to persist clipboard content — child process exits independently - -## Design Patterns - -- **Trait-based abstraction**: `FileResolver` enables two completely different I/O backends (std::fs, WASI) from the same core -- **BFS with context-sensitive dedup**: `PreprocContext` stack tracks conditional directive state so `#include` inside `#ifdef`/`#else` branches is correctly handled -- **Reusable state machine**: `CompressorState` can be reused across multiple tree walks, tracking compound directive nesting and body newline insertion diff --git a/.claude/project-tracker/data-model.md b/.claude/project-tracker/data-model.md deleted file mode 100644 index 6037fa4..0000000 --- a/.claude/project-tracker/data-model.md +++ /dev/null @@ -1,88 +0,0 @@ -# Data Model - -## Core Types - -### `FileResolver` trait (resolver.rs) -``` -FileResolver -├── resolve(includer_path, include_path) to Result -└── read_content(resolved_path) to Result -``` - -The central I/O boundary. Two implementations: -- `FsResolver` (CLI): `std::fs::canonicalize` + `read_to_string` -- `WasiFsResolver` (VSCode): WASI filesystem paths - -### Include types (parser.rs) -``` -Include<'a> -├── Local(&'a str) // #include "path" -└── System(&'a str) // #include -``` - -### Preproc directive stack (expander.rs) -``` -PreprocDirective -├── If(Subject) -├── Ifdef(Subject) -├── Ifndef(Subject) -├── Elif(Subject) -├── Elifdef(Subject) -└── Else - -PreprocContext(Vec) -``` - -The `Subject` is a `Vec` — the token sequence extracted from the conditionals argument for structural equivalence comparison. - -### ExpandState (expander.rs) -``` -ExpandState -├── completed: HashSet<(PathBuf, PreprocContext)> -│ Files + context pairs already expanded (dedup key) -├── expanding: HashSet -│ Cycle detection stack -└── tree_cache: HashMap -│ Parsed AST cache -``` - -### CompressorState (compressor.rs) -``` -CompressorState -├── output: String // Accumulated compressed output -├── prev_last: Option // Last emitted char (identifier spacing) -├── compound_depth: usize // #ifdef / #if nesting depth -└── body_nl_counter: Option - Sibling counter for body newline insertion -``` - -## Config Model - -### CLI config (texpand-cli/config.rs) -```toml -# ~/.config/texpand.toml -include_paths = ["./templates", "~/algo/cpp_lib"] -default_compress = false -``` - -### VSCode settings -| Key | Type | Default | -|-----|------|---------| -| `texpand.includePaths` | `string[]` | `["./"]` | -| `texpand.defaultCompression` | `boolean` | `false` | -| `texpand.outputMode` | `"clipboard"` or `"newFile"` | `"clipboard"` | -| `texpand.saveBeforeExpansion` | `boolean` | `true` | - -## VSCode Result Envelope -``` -ExpandResult (JSON to stdout) -├── success: bool -├── data: Option // Expanded output -└── error: Option // Error message -``` - -## Key Invariants - -- `texpand-core` must never call `std::fs` or `std::io` — all file data enters through `FileResolver` -- `PreprocContext` is purely structural — no actual macro evaluation; same token sequence = same context -- Dedup key `(PathBuf, PreprocContext)` means the same file CAN appear multiple times if included under different `#ifdef` branches diff --git a/.claude/project-tracker/deployment.md b/.claude/project-tracker/deployment.md deleted file mode 100644 index 81403fe..0000000 --- a/.claude/project-tracker/deployment.md +++ /dev/null @@ -1,59 +0,0 @@ -# Deployment - -## Build Artifacts - -### CLI Binary -- **Format**: Single platform-native executable -- **Targets**: x86_64 Linux, ARM64 Linux, x86_64 macOS, ARM64 macOS, x86_64 Windows -- **Build**: `cargo build -p texpand-cli --release --target ` -- **Size**: ~3-5 MB per binary (stripped, LTO) -- **Install**: `cargo install --path texpand-cli` or download from GitHub Releases - -### VSCode Extension -- **Format**: `.vsix` package -- **Contents**: WASM binary (wasi target) + bundled TypeScript -- **Build**: - 1. `cargo build -p texpand-vscode --target wasm32-wasip1 --release` - 2. `wasm-opt` for optimization - 3. `esbuild` for TypeScript bundling - 4. `vsce package` for .vsix -- **Published**: GitHub Releases (not on Marketplace) - -## Release Process - -Trigger: push tag `v*` to GitHub. - -``` -Release workflow (release.yml): -├── build-cli (matrix: 5 targets) -│ ├── Install Rust + cross-compilers -│ ├── cargo build --release -│ ├── Rename to texpand- -│ └── Upload artifact -├── build-vsix (ubuntu-latest) -│ ├── Install WASI SDK 33 -│ ├── npm ci -│ ├── npm run vscode:prepublish (WASM + esbuild) -│ ├── vsce package -│ └── Upload artifact -└── release (after both complete) - ├── Download all artifacts - └── Create GitHub Release with generated notes -``` - -## Environments - -| Environment | Distribution | Update Mechanism | -|-------------|-------------|-----------------| -| Local dev | `cargo build` | Manual rebuild | -| End-user CLI | GitHub Releases / `cargo install` | Manual | -| VSCode | `.vsix` sideload | Manual install from VSIX | - -## Health Checks & Monitoring - -N/A — local CLI tool with no server component. The VSCode extension writes diagnostic messages to stderr (visible in output channel). No telemetry or crash reporting. - -## Rollback - -- CLI: reinstall previous version via `cargo install --version ` or download old release binary -- VSCode: Install from VSIX with previous version, or uninstall/reinstall diff --git a/.claude/project-tracker/implementation.md b/.claude/project-tracker/implementation.md deleted file mode 100644 index c90e5ce..0000000 --- a/.claude/project-tracker/implementation.md +++ /dev/null @@ -1,79 +0,0 @@ -# Implementation Details - -## Entry Points - -| Crate | Entry | Mechanism | -|-------|-------|-----------| -| texpand-cli | `main()` in `main.rs` | `clap::Parser` to `expand()` | -| texpand-vscode | `main()` in `src/main.rs` | WASI process, env-var config to `expand()` to JSON stdout | -| VSCode TS | `extension.ts` | VSCode activation events to WASM lifecycle to result display | - -## Request Trace (CLI) - -``` -cli/main.rs:main() - → Cli::parse() # clap argument parsing - → config::TexpandConfig::load() # optional ~/.config/texpand.toml - → FsResolver::new() # std::fs-backed resolver - → expander::expand() # recursive BFS expansion - → parser::parse_source() # first parse - → expand_recursive() # DFS AST walk - → classify_include() # identify Local vs System includes - → FileResolver::resolve() # resolve #include path - → FileResolver::read_content() # read included file - → expand_recursive() # recurse into dependency - → CompressorState (optional) # token-level compression - → output (clipboard/file/stdout) -``` - -## Key Algorithms - -### BFS Expansion with Preproc Context - -In `expander.rs`, `expand_recursive()` walks the AST in DFS order. For each node: - -1. **`preproc_include`**: resolve local includes via `FileResolver`, recurse, inline result. System includes preserved as-is. -2. **`#pragma once`**: stripped entirely. -3. **Compound conditionals** (`#ifdef`, `#ifndef`, `#if`): push `PreprocDirective` onto a context stack. The context tracks which conditional branch we are in. -4. **Dedup key**: `(file_path, PreprocContext)` — the same file can be expanded in different conditional contexts. - -### Cycle Detection - -Uses a `expanding: HashSet` stack. If a file is encountered while already on the stack, the full cycle path is reported in the error. This is set-based, not graph-based — linear in expansion depth. - -### Token-Level Compression - -`compressor.rs` walks AST leaves only: -- Comments are skipped entirely (tree-sitter `"comment"` node kind) -- Space inserted between adjacent identifier characters (`[a-zA-Z0-9_]`) -- Before/after `#` preprocessor directives: force newline -- Compound preproc bodies (`#ifdef ... #endif`): insert newline before body -- `literal_suffix` nodes: no space inserted (preserves `123_km`) -- `#define` name field: trailing space forced (prevents `#define FOO"abc"`) - -## Error Handling - -- `anyhow::Result` throughout both CLI and WASM frontends -- `.with_context()` for descriptive error messages -- Cycle detection produces a human-readable path -- VSCode WASM wraps errors in JSON `{ success: false, error: "..." }` envelope - -## Testing Strategy - -- **Unit tests**: inline in each source file (`#[cfg(test)] mod tests`) -- **Integration tests**: `tests/` directory with `FixtureResolver` providing in-memory file content -- **Fixtures**: `fixtures/` directory with real C/C++ files for CLI end-to-end -- **Key test scenarios**: - - Basic expansion / transitive dependencies / diamond deps - - Circular dependency detection - - Compression (comments, identifiers, preproc, defines, user-defined literals) - - Conditional includes inside `#ifdef`/`#else`/`#endif` blocks - - System include preservation - - Edge cases: empty source, only comments, nested ifdefs - -## Performance Considerations - -- **AST caching**: Parsed trees cached in `HashMap` to avoid re-parsing -- **Capacity hints**: Output strings pre-allocated with `String::with_capacity` -- **Release profile**: LTO + strip + panic=abort minimizes binary size -- **No unnecessary allocation**: `CompressorState` keeps a reusable buffer diff --git a/.claude/project-tracker/modules/cli.md b/.claude/project-tracker/modules/cli.md deleted file mode 100644 index 2a659f3..0000000 --- a/.claude/project-tracker/modules/cli.md +++ /dev/null @@ -1,47 +0,0 @@ -# Module: texpand-cli - -**Path**: `texpand-cli/` -**Type**: Binary crate (CLI frontend) -**Entry**: `src/main.rs` — `fn main()` - -## Responsibility - -Provides the command-line interface for texpand. Reads files from the local filesystem via `FsResolver`, handles config from `~/.config/texpand.toml`, and manages output (stdout, file, clipboard). - -## Key Components - -### `FsResolver` (main.rs:54-89) -Implementation of `FileResolver` using `std::fs`: -- Absolute paths: resolved directly via `canonicalize()` -- Relative paths: first checked relative to includer directory, then against configured `include_paths` -- `read_content()`: delegates to `std::fs::read_to_string()` - -### CLI Parser (main.rs:15-48) -Clap derive-based parser with flags: -- `-c`/`--compress` and `--no-compress` — overrides config -- `-i`/`--include` — search paths (repeatable, overrides config) -- `-o`/`--output` — file output -- `-C`/`--clipboard` — clipboard output (mutually exclusive with `-o`) -- `--config` — custom config path -- Stdin support via `-` as input path - -### Clipboard (main.rs:91-118) -- **Linux**: forks a child process that holds the clipboard data alive (arboard clipboard vanishes when process exits) -- **Other platforms**: direct `set_text()` call - -### Config (config.rs) -`TexpandConfig` deserialized from TOML: -- `include_paths: Vec` -- `default_compress: bool` -- Located at `~/.config/texpand.toml` (respects `XDG_CONFIG_HOME`) - -## Dependencies - -| Crate | Usage | -|-------|-------| -| `texpand-core` | Core expansion library | -| `clap 4` | CLI argument parsing | -| `serde 1` | Config deserialization | -| `toml 1` | Config file parsing | -| `arboard 3` | Clipboard access | -| `nix 0` | Linux fork syscall | diff --git a/.claude/project-tracker/modules/core.md b/.claude/project-tracker/modules/core.md deleted file mode 100644 index fb89219..0000000 --- a/.claude/project-tracker/modules/core.md +++ /dev/null @@ -1,39 +0,0 @@ -# Module: texpand-core - -**Path**: `texpand-core/` -**Type**: Library crate (I/O-free core) -**Entry**: `src/lib.rs` — re-exports 4 public modules - -## Responsibility - -Central processing engine for C/C++ include expansion. Must never call `std::fs` or `std::io` directly — all file I/O goes through the `FileResolver` trait. - -## Modules - -| Module | File | Lines | Key Items | -|--------|------|-------|-----------| -| `resolver` | `resolver.rs` | 26 | `FileResolver` trait | -| `parser` | `parser.rs` | 131 | `parse_source()`, `Include`, `extract_all_includes()` | -| `expander` | `expander.rs` | 525 | `expand()`, `ExpandOptions`, `PreprocContext`, `ExpandState` | -| `compressor` | `compressor.rs` | 552 | `CompressorState`, `compress()`, `compress_stripped()` | - -## Dependencies - -``` -tree-sitter 0.24 -tree-sitter-cpp 0.23 (C/C++ grammar for preproc-aware AST) -serde 1 + derive (for ExpandResult serialization) -anyhow 1 (error propagation) -``` - -## Unique Patterns - -- **PreprocContext dedup**: The `completed` set is `HashSet<(PathBuf, PreprocContext)>` — files are NOT deduplicated globally, only within the same preprocessor conditional context. This is the core correctness guarantee. -- **Trait-based I/O isolation**: `FileResolver` is the only way data enters the core. The trait has exactly two methods: `resolve()` (path resolution) and `read_content()` (file content). -- **Reusable CompressorState**: The state machine tracks `prev_last` char for identifier spacing, `compound_depth` for preproc nesting, and `body_nl_counter` for body newline insertion. - -## Tests - -- Unit tests inline in each module under `#[cfg(test)]` -- Integration tests in `tests/` using `FixtureResolver` (in-memory file map) -- Test data in `fixtures/` (real C/C++ files for CLI e2e) diff --git a/.claude/project-tracker/modules/vscode-extension.md b/.claude/project-tracker/modules/vscode-extension.md deleted file mode 100644 index 00a0606..0000000 --- a/.claude/project-tracker/modules/vscode-extension.md +++ /dev/null @@ -1,88 +0,0 @@ -# Module: texpand-vscode - -**Path**: `texpand-vscode/` -**Type**: WASM binary + TypeScript extension -**Entry**: Rust `src/main.rs`, TypeScript `extension/src/extension.ts` - -## Responsibility - -VSCode extension that provides expand-in-place and expand-to-clipboard functionality for C/C++ files. Runs the core Rust engine as a WASI process — no native binary dependency. - -## Rust WASM Layer (`src/main.rs`) - -WASI process entry point. Reads configuration from environment variables: - -| Env Var | Purpose | -|---------|---------| -| `TEXPAND_ENTRY_PATH` | Entry source file path | -| `TEXPAND_COMPRESS` | `"true"` to enable compression | -| `TEXPAND_INCLUDE_PATHS` | Comma-separated include search paths | - -**WasiFsResolver**: Implements `FileResolver` using WASI filesystem access. Resolves includes relative to the includers directory, then against configured prefix paths. - -**Output**: Writes JSON to stdout: -- Success: `{ "success": true, "data": "" }` -- Error: `{ "success": false, "error": "" }` - -## TypeScript Extension (`extension/src/`) - -### Files -| File | Purpose | -|------|---------| -| `extension.ts` | VSCode activation, command registration, result handling | -| `wasm.ts` | WASI process lifecycle management | - -### Activation -Triggers on `onLanguage:c` and `onLanguage:cpp` events. - -### Commands -| Command | Behavior | -|---------|----------| -| `texpand.expandDefault` | Uses `texpand.outputMode` setting | -| `texpand.expandAndCopy` | Forces clipboard output | -| `texpand.expandToNewFile` | Creates `*.expanded.cpp` file | - -### WASM Lifecycle (`wasm.ts`) -1. Resolves WASM binary path relative to extension -2. Creates `WasiProcess` with mapped workspace directories -3. Sets environment variables from extension settings -4. Reads stdout until process exits -5. Parses JSON result and handles errors - -### Settings (contributes to `package.json`) -| Key | Type | Default | -|-----|------|---------| -| `texpand.includePaths` | `string[]` | `["./"]` | -| `texpand.defaultCompression` | `boolean` | `false` | -| `texpand.outputMode` | `clipboard` or `newFile` | `"clipboard"` | -| `texpand.saveBeforeExpansion` | `boolean` | `true` | - -## Build Pipeline - -``` -Cargo.toml - ↓ cargo build --target wasm32-wasip1 --release -WASM binary (target/wasm32-wasip1/release/) - ↓ wasm-opt -Optimized WASM (pkg/texpand-vscode.wasm) - ↓ esbuild bundling -dist/extension.js - ↓ vsce package -texpand-vscode-*.vsix -``` - -## Dependencies - -### Rust -| Crate | Usage | -|-------|-------| -| `texpand-core` | Core expansion | -| `serde` + `serde_json` | JSON output | -| `anyhow` | Error handling | - -### TypeScript -| Package | Usage | -|---------|-------| -| `@vscode/wasm-wasi` | WASI process host | -| `@vscode/vsce` | VSIX packaging | -| `esbuild` | TypeScript bundling | diff --git a/.claude/project-tracker/progress.md b/.claude/project-tracker/progress.md deleted file mode 100644 index aba340d..0000000 --- a/.claude/project-tracker/progress.md +++ /dev/null @@ -1,47 +0,0 @@ -# Progress - -## Current Phase - -v0.2.0 — Stable release with dual CLI + VSCode support. - -## Completed Features - -- [x] Tree-sitter based C/C++ parsing with preprocessor support -- [x] Local `#include "..."` expansion via BFS with cycle detection -- [x] System `#include <...>` preservation in output -- [x] `#pragma once` stripping during expansion -- [x] Context-sensitive dedup: same file in different `#ifdef` branches re-expanded correctly -- [x] Token-level semantic-safe compression (comment removal, identifier spacing, preproc newlines) -- [x] User-defined literal preservation in compressor (`123_km` stays intact) -- [x] `#define` space preservation (macro name to replacement separation) -- [x] CLI frontend with clap (stdin, file output, clipboard, config file) -- [x] Linux clipboard fork daemon for paste persistence -- [x] VSCode extension via WASM-WASI (3 commands, virtual filesystem, l10n) -- [x] Cross-platform CI (5 targets) + VSCode VSIX packaging -- [x] Config file (`~/.config/texpand.toml`) with include paths -- [x] l10n support with Chinese locale - -## Known Issues & Technical Debt - -- No `-isystem` vs `-I` distinction in include resolution -- No diagnostic output mode (e.g., `--verbose` showing which files were expanded) -- Compression edge: some C++ constructs may need manual review (e.g., string literal concatenation) -- No benchmark suite for expansion performance -- Test fixtures are hand-written — could benefit from fuzzing - -## Roadmap - -### Near-term (v0.3.0) -- [ ] Support for `-isystem` include paths (system vs local include resolution) -- [ ] Verbose/diagnostic output mode -- [ ] Recursive directory scanning for include paths - -### Medium-term -- [ ] Support for CMake `compile_commands.json` include path extraction -- [ ] `#pragma once` vs include guard heuristic detection -- [ ] WASM size optimization - -### Future -- [ ] Language server protocol (LSP) integration -- [ ] Online Judge auto-submit feature -- [ ] Emacs integration diff --git a/.claude/project-tracker/stack.md b/.claude/project-tracker/stack.md deleted file mode 100644 index d2fa8c7..0000000 --- a/.claude/project-tracker/stack.md +++ /dev/null @@ -1,49 +0,0 @@ -# Technology Stack - -## Language & Runtime - -| Layer | Technology | Version | Rationale | -|-------|-----------|---------|-----------| -| Core library | Rust | edition 2024 | Performance, safety, WASM target support | -| CLI binary | Rust | same | No runtime overhead, easy distribution | -| VSCode extension | TypeScript + WASM | Node 22 | WASM-WASI integration, no native binary needed | - -## Key Frameworks & Libraries - -### texpand-core - -| Library | Purpose | Why | -|---------|---------|-----| -| `tree-sitter` 0.24 | C/C++ parsing | Incremental, robust AST generation for real-world C/C++ code | -| `tree-sitter-cpp` 0.23 | C/C++ grammar | Supports preprocessor directives as AST nodes (essential for include analysis) | -| `serde` 1 | Serialization | Result JSON for VSCode frontend | - -### texpand-cli - -| Library | Purpose | Why | -|---------|---------|-----| -| `clap` 4 | CLI argument parsing | Derive-based, compile-time checks, subcommand support | -| `toml` 1 | Config file parsing | Standard format for `~/.config/texpand.toml` | -| `arboard` 3 | Clipboard access | Cross-platform clipboard (forks on Linux for persistence) | -| `nix` 0 | Fork syscall | Linux clipboard daemon forking on Linux | - -### texpand-vscode - -| Library | Purpose | Why | -|---------|---------|-----| -| `serde_json` 1 | JSON output | Structured result for TypeScript host | - -(VSCode side uses `@vscode/wasm-wasi` for WASM process lifecycle and `esbuild` for bundling.) - -## Storage - -- **Config file**: `~/.config/texpand.toml` (TOML) — optional, CLI only -- **VSCode settings**: `settings.json` under `texpand.*` keys — extension side -- No database layer — the tool operates on ephemeral source files - -## Infrastructure - -- **CI**: GitHub Actions (push + PR triggers) -- **Release**: GitHub Releases with cross-platform CLI binaries + `.vsix` -- **WASI SDK**: wasi-sdk 33 for WASM compilation (wasm32-wasip1 target) -- **Cross-compilation**: gcc-aarch64-linux-gnu for ARM64 Linux builds diff --git a/.claude/project-tracker/toolchain.md b/.claude/project-tracker/toolchain.md deleted file mode 100644 index cafa2ba..0000000 --- a/.claude/project-tracker/toolchain.md +++ /dev/null @@ -1,81 +0,0 @@ -# Toolchain - -## Build System - -Cargo workspace with 3 members. Release profile: LTO + panic=abort + strip. - -```bash -# Build all crates -cargo build --workspace - -# Build specific crate -cargo build -p texpand-core -cargo build -p texpand-cli -cargo build -p texpand-vscode --target wasm32-wasip1 --release -``` - -## Linting & Formatting - -- **Formatter**: `cargo fmt --all` (rustfmt, 100-char line width, 4-space indent) -- **Linter**: `cargo clippy --workspace -- -D warnings` (deny warnings as errors) - -## Testing - -- **Framework**: built-in `#[test]` with `#[cfg(test)]` modules -- **Unit tests**: inline in each source file under `mod tests` -- **Integration tests**: `texpand-core/tests/` — use `FixtureResolver` (in-memory file map) -- **Coverage target**: not explicitly configured (no coverage CI step) -- **Run**: `cargo test --workspace` - -``` -texpand-core/tests/ -├── common.rs # Shared test utilities (FixtureResolver) -├── test_basic_expansion.rs -├── test_circular_dep.rs -├── test_compression.rs -├── test_conditional_includes.rs -├── test_edge_cases.rs -└── test_system_include.rs -``` - -## CI/CD Pipeline - -### CI (`.github/workflows/ci.yml`) -Triggers on push to any branch and PRs: - -| Job | Tool | Purpose | -|-----|------|---------| -| `check` | `cargo check` | Verify compilation | -| `fmt` | `cargo fmt -- --check` | Formatting compliance | -| `clippy` | `cargo clippy -- -D warnings` | Lint enforcement | -| `test` | `cargo test --workspace` | All tests | -| `typecheck` | `tsc --noEmit` | VSCode extension types | - -### Release (`.github/workflows/release.yml`) -Triggers on `v*` tag push: - -- **CLI binaries**: 5 targets (x86_64 Linux/ARM64 Linux/x86_64 macOS/ARM64 macOS/x86_64 Windows) -- **VSIX package**: WASM build + esbuild bundle + `vsce package` -- **GitHub Release**: auto-generated release notes, all artifacts attached - -## Dev Environment Prerequisites - -| Tool | Purpose | -|------|---------| -| Rust stable toolchain | Core compilation | -| wasm32-wasip1 target | VSCode WASM build | -| wasi-sdk 33 | WASM C/C++ linker | -| Node.js 22 | VSCode extension build | -| wasm-opt | WASM binary optimization | -| cargo-llvm-cov (optional) | Coverage reporting | - -## Environment Variables - -| Variable | Used By | Purpose | -|----------|---------|---------| -| `TEXPAND_ENTRY_PATH` | texpand-vscode WASM | Entry source file path | -| `TEXPAND_COMPRESS` | texpand-vscode WASM | Enable compression flag | -| `TEXPAND_INCLUDE_PATHS` | texpand-vscode WASM | Comma-separated include search paths | -| `XDG_CONFIG_HOME` | texpand-cli | Config file location override | -| `CC_wasm32_wasip1` | .cargo/config.toml | WASI SDK clang path | -| `CXX_wasm32_wasip1` | .cargo/config.toml | WASI SDK clang++ path | diff --git a/.project-tracker/.state.json b/.project-tracker/.state.json new file mode 100644 index 0000000..1a518f0 --- /dev/null +++ b/.project-tracker/.state.json @@ -0,0 +1,204 @@ +{ + "files": { + "INDEX.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".cargo/config.toml", + ".github/workflows/ci.yml", + ".github/workflows/release.yml", + "Cargo.lock", + "Cargo.toml", + "README.md", + "texpand-cli/Cargo.toml", + "texpand-core/Cargo.toml", + "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/package.json" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "api.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "README.md", + "texpand-cli/src/config.rs", + "texpand-cli/src/main.rs", + "texpand-vscode/extension/package.json", + "texpand-vscode/extension/src/extension.ts", + "texpand-vscode/extension/src/wasm.ts", + "texpand-vscode/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "architecture.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".claude/CLAUDE.md", + "Cargo.toml", + "texpand-cli/src/config.rs", + "texpand-cli/src/main.rs", + "texpand-core/src/compressor.rs", + "texpand-core/src/expander.rs", + "texpand-core/src/lib.rs", + "texpand-core/src/parser.rs", + "texpand-core/src/resolver.rs", + "texpand-vscode/extension/src/extension.ts", + "texpand-vscode/extension/src/wasm.ts", + "texpand-vscode/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "conventions.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".claude/CLAUDE.md", + "Cargo.toml", + "texpand-cli/Cargo.toml", + "texpand-core/Cargo.toml", + "texpand-core/src/compressor.rs", + "texpand-core/src/expander.rs", + "texpand-core/src/lib.rs", + "texpand-core/src/parser.rs", + "texpand-core/src/resolver.rs", + "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/eslint.config.mjs", + "texpand-vscode/extension/src/extension.ts", + "texpand-vscode/extension/src/wasm.ts", + "texpand-vscode/extension/tsconfig.json" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "data-model.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "Cargo.toml", + "texpand-cli/src/config.rs", + "texpand-cli/src/main.rs", + "texpand-core/src/compressor.rs", + "texpand-core/src/expander.rs", + "texpand-core/src/lib.rs", + "texpand-core/src/parser.rs", + "texpand-core/src/resolver.rs", + "texpand-vscode/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "deployment.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".github/workflows/release.yml", + "Cargo.toml", + "texpand-cli/Cargo.toml", + "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/package-lock.json", + "texpand-vscode/extension/package.json" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "implementation.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "README.md", + "texpand-cli/src/config.rs", + "texpand-cli/src/main.rs", + "texpand-core/src/compressor.rs", + "texpand-core/src/expander.rs", + "texpand-core/src/lib.rs", + "texpand-core/src/parser.rs", + "texpand-core/src/resolver.rs", + "texpand-core/tests/common.rs", + "texpand-core/tests/test_basic_expansion.rs", + "texpand-core/tests/test_circular_dep.rs", + "texpand-core/tests/test_compression.rs", + "texpand-core/tests/test_conditional_includes.rs", + "texpand-core/tests/test_edge_cases.rs", + "texpand-core/tests/test_system_include.rs", + "texpand-vscode/extension/src/extension.ts", + "texpand-vscode/extension/src/wasm.ts", + "texpand-vscode/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "modules/cli.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "README.md", + "texpand-cli/Cargo.toml", + "texpand-cli/src/config.rs", + "texpand-cli/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "modules/core.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "texpand-core/Cargo.toml", + "texpand-core/src/compressor.rs", + "texpand-core/src/expander.rs", + "texpand-core/src/lib.rs", + "texpand-core/src/parser.rs", + "texpand-core/src/resolver.rs", + "texpand-core/tests/common.rs", + "texpand-core/tests/test_basic_expansion.rs", + "texpand-core/tests/test_circular_dep.rs", + "texpand-core/tests/test_compression.rs", + "texpand-core/tests/test_conditional_includes.rs", + "texpand-core/tests/test_edge_cases.rs", + "texpand-core/tests/test_system_include.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "modules/vscode.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/README.md", + "texpand-vscode/extension/README.zh-CN.md", + "texpand-vscode/extension/l10n/bundle.l10n.json", + "texpand-vscode/extension/l10n/bundle.l10n.zh-cn.json", + "texpand-vscode/extension/package.json", + "texpand-vscode/extension/package.nls.json", + "texpand-vscode/extension/package.nls.qps-ploc.json", + "texpand-vscode/extension/package.nls.zh-cn.json", + "texpand-vscode/extension/src/extension.ts", + "texpand-vscode/extension/src/wasm.ts", + "texpand-vscode/src/main.rs" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "progress.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [], + "updated": "2026-05-29T14:24:46Z" + }, + "stack.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".github/workflows/ci.yml", + ".github/workflows/release.yml", + "Cargo.lock", + "Cargo.toml", + "texpand-cli/Cargo.toml", + "texpand-core/Cargo.toml", + "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/package.json" + ], + "updated": "2026-05-29T14:24:46Z" + }, + "toolchain.md": { + "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "matched_paths": [ + ".cargo/config.toml", + ".claude/CLAUDE.md", + ".github/workflows/ci.yml", + ".github/workflows/release.yml", + "Cargo.lock", + "Cargo.toml", + "texpand-vscode/extension/eslint.config.mjs", + "texpand-vscode/extension/package.json", + "texpand-vscode/extension/tsconfig.json" + ], + "updated": "2026-05-29T14:24:46Z" + } + }, + "version": 1 +} diff --git a/.project-tracker/INDEX.md b/.project-tracker/INDEX.md new file mode 100644 index 0000000..fd32db5 --- /dev/null +++ b/.project-tracker/INDEX.md @@ -0,0 +1,74 @@ +--- +sources: + - "Cargo.toml" + - "Cargo.lock" + - "README.md" + - ".github/workflows/*.yml" + - ".cargo/config.toml" + - "texpand-*/Cargo.toml" + - "texpand-vscode/extension/package.json" +--- + +# PROJECT: Template-Expand + +Rust monorepo for expanding local C/C++ template headers into a single-file submission, with both CLI and VS Code frontends. + +## Table of Contents + +- [Stack](stack.md) — Technology choices and dependencies +- [Toolchain](toolchain.md) — Build, test, CI/CD, dev setup +- [Architecture](architecture.md) — Module layout and data flow +- [Conventions](conventions.md) — Coding standards, naming rules, architectural rules +- [Progress](progress.md) — Current status and roadmap +- [Implementation](implementation.md) — Key implementation details +- [Data Model](data-model.md) — Project data/storage model +- [API](api.md) — Public API surface +- [Deployment](deployment.md) — Build and release packaging +- [Modules](modules/core.md), [CLI](modules/cli.md), [VS Code](modules/vscode.md) — Per-component notes + +## Tech Stack Summary + +| Layer | Technology | Version | +|-------|-----------|---------| +| Language | Rust | Edition 2024 | +| Frontend | TypeScript for VS Code extension | TypeScript 5.9.x | +| Parser | tree-sitter + tree-sitter-cpp | 0.24 / 0.23 | +| Packaging | Cargo + npm + VSCE | lockfile-managed | +| CI/CD | GitHub Actions | — | + +- `texpand-core` is intentionally I/O-free and exposes the reusable expansion engine. +- `texpand-cli` provides local filesystem, config, and clipboard integration. +- `texpand-vscode` compiles the Rust frontend to WASI/WASM and wraps it in a VS Code extension. +- Workspace crate manifests and extension package metadata are currently aligned on release version `0.2.2`. +- Releases publish native CLI binaries and a `.vsix` extension artifact. + +## Quick Reference Commands + +```bash +# Build +cargo build --workspace + +# Test +cargo test --workspace + +# Run +cargo run -p texpand-cli -- fixtures/basic/main.cpp + +# Lint +cargo clippy --workspace -- -D warnings +``` + +## Project Map + +- `texpand-core/` — parser, resolver abstraction, recursive expander, and compression logic. +- `texpand-cli/` — command-line interface, config loading, filesystem-backed resolver, clipboard output. +- `texpand-vscode/` — Rust WASI binary plus the TypeScript VS Code extension wrapper. +- `fixtures/` — C/C++ sample trees used for manual and automated verification. +- `.github/workflows/` — CI validation and tagged release automation. + +## Tracking Exclusions + +- `target/**` -- build outputs and generated docs are not source-of-truth inputs. +- `texpand-vscode/extension/node_modules/**` -- vendored dependencies are reproducible from `package-lock.json`. +- `texpand-vscode/extension/.vscode/**` -- local VS Code workspace settings are development-only and not architecture-tracked. +- `.claude/settings.local.json` -- local harness/editor preferences are intentionally user-specific. diff --git a/.project-tracker/api.md b/.project-tracker/api.md new file mode 100644 index 0000000..acc503c --- /dev/null +++ b/.project-tracker/api.md @@ -0,0 +1,56 @@ +--- +sources: + - "texpand-cli/src/*.rs" + - "texpand-vscode/src/*.rs" + - "texpand-vscode/extension/src/*.ts" + - "texpand-vscode/extension/package.json" + - "README.md" +--- + +# API Reference + +N/A — the repository exposes a CLI and a VS Code command surface, not an HTTP or RPC API. + +## Endpoints + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| N/A | N/A | N/A | No network endpoints are implemented | + +## Request / Response + +### CLI surface + +- Input: source file path or `-` for stdin, plus flags for include paths, compression, output file, clipboard, and config. +- Output: expanded C/C++ text to stdout, a file, or the clipboard. + +### VS Code WASI process contract + +**Request:** + +- Environment variables: + - `TEXPAND_ENTRY_PATH` + - `TEXPAND_COMPRESS` + - `TEXPAND_INCLUDE_PATHS` + +**Response:** + +```json +{ + "success": true, + "data": "expanded C/C++ text", + "error": null +} +``` + +The extension surface remains local-only; the current manifest version is `0.2.2`, but the request/response contract is unchanged by this maintenance update. + +## Rate Limiting + +| Window | Limit | Behavior | +|--------|-------|----------| +| N/A | N/A | No network service exists | + +## Pagination + +Not applicable. diff --git a/.project-tracker/architecture.md b/.project-tracker/architecture.md new file mode 100644 index 0000000..f961455 --- /dev/null +++ b/.project-tracker/architecture.md @@ -0,0 +1,69 @@ +--- +sources: + - "Cargo.toml" + - "texpand-core/src/*.rs" + - "texpand-cli/src/*.rs" + - "texpand-vscode/src/*.rs" + - "texpand-vscode/extension/src/*.ts" + - ".claude/CLAUDE.md" +--- + +# Architecture + +## Overview + +The repository is a small monorepo: one pure Rust library plus two frontends that provide environment-specific file access and output handling. + +```text ++---------------------------+ +-------------------------------+ +| User source: main.cpp | | VS Code workspace / local FS | ++-------------+-------------+ +---------------+---------------+ + | | + v v + +-------+--------+ +--------+--------+ + | Frontend layer | | FileResolver | + | CLI or WASM +------------------------>+ implementation | + +-------+--------+ +--------+--------+ + | | + v | + +-------+-------------------------------------------+--------+ + | texpand-core | + | parser.rs -> expander.rs -> compressor.rs | + +--------------------------+--------------------------------+ + | + v + Expanded single-file C/C++ output +``` + +## Module Breakdown + +| Module / Crate | Responsibility | Key types / exports | +|---------------|---------------|--------------------| +| `texpand-core` | Parse C/C++ code, resolve include graphs, inline local headers, preserve or compress output | `expand`, `ExpandOptions`, `FileResolver`, `CompressorState` | +| `texpand-cli` | Native command entry point, config loading, filesystem resolution, clipboard/file/stdout output | `Cli`, `FsResolver`, `TexpandConfig` | +| `texpand-vscode` | Rust WASI entry point that adapts env vars to `texpand-core` and emits JSON | `WasiFsResolver`, `ExpandResult` | +| `texpand-vscode/extension` | VS Code activation, command registration, WASI process orchestration, clipboard/new-file UX | `activate`, `runExpansion`, `expandWithProcess` | + +## Data Flow + +1. A frontend receives the entry file path and user options such as compression and include paths. +2. The frontend builds a `FileResolver` implementation appropriate to its runtime. +3. `texpand-core::expand` parses the entry file with tree-sitter and recursively walks `#include` directives. +4. Local includes are resolved and expanded once per preprocessing context; system includes are preserved in output. +5. Optional compression rewrites the output by traversing syntax leaves instead of using text-level minification, with shared leaf-emission rules between standalone compression and expansion-time compression. +6. The frontend writes the final text to stdout, clipboard, a file, or a JSON payload. + +## Design Patterns + +- Resolver abstraction -- `FileResolver` keeps `texpand-core` independent from native filesystem and VS Code workspace APIs. +- Context-sensitive deduplication -- include expansion keys combine file path and preprocessor context so the same header can appear in different conditional branches when required. +- Cached parsing -- `ExpandState` stores parsed trees by path within a run to avoid repeated tree-sitter work. +- Shared token emission -- compressor and compressed-expansion paths reuse the same leaf emission rules so macro spacing semantics stay consistent. +- Thin adapters -- CLI and VS Code crates mainly translate environment concerns into the shared core API. + +## Security Boundaries + +- Only local/project files are read; there is no network API surface in the codebase. +- The extension constrains WASI file access through explicit mount points rather than arbitrary host access. +- CLI configuration is parsed from TOML and command flags instead of executing user-provided scripts. +- Error messages carry file names and parse context, but there is no secrets/config credential surface in the repo. diff --git a/.project-tracker/conventions.md b/.project-tracker/conventions.md new file mode 100644 index 0000000..3bfcd19 --- /dev/null +++ b/.project-tracker/conventions.md @@ -0,0 +1,89 @@ +--- +sources: + - ".claude/CLAUDE.md" + - "Cargo.toml" + - "texpand-core/Cargo.toml" + - "texpand-cli/Cargo.toml" + - "texpand-vscode/Cargo.toml" + - "texpand-vscode/extension/eslint.config.mjs" + - "texpand-vscode/extension/tsconfig.json" + - "texpand-vscode/extension/src/*.ts" + - "texpand-core/src/*.rs" +--- + +# Project Conventions + +> Agents MUST read and follow these conventions. + +## Coding Conventions + +| Aspect | Rule | Config | +|--------|------|--------| +| Formatter | Use `rustfmt` defaults for Rust sources | default `rustfmt` | +| Linter | `cargo clippy --workspace -- -D warnings` must pass | workspace/CI | +| Max line length | Not explicitly configured | — | +| Indentation | Rust uses standard formatter output; TypeScript codebase uses 4-space indentation in practice | source files | +| Quote style | Rust strings normal; TypeScript uses single quotes | `texpand-vscode/extension/src/*.ts` | +| Semicolons | Required in TypeScript; standard Rust statement style | source files | +| Trailing commas | Preferred where formatter inserts them | source files | + +## Naming Conventions + +| Category | Convention | Example | +|----------|-----------|---------| +| Files / modules | Rust modules use `snake_case`; docs use kebab/flat names | `expander.rs`, `test_basic_expansion.rs` | +| Variables | `snake_case` in Rust, `camelCase` in TypeScript | `include_paths`, `outputMode` | +| Constants | `UPPER_SNAKE_CASE` | `WASI_SDK_VERSION` | +| Functions / methods | `snake_case` in Rust, `camelCase` in TypeScript | `extract_subject`, `showConfigQuickPick` | +| Types / classes / components | `PascalCase` | `ExpandOptions`, `FixtureResolver` | + +## Architectural Rules + +- `texpand-core` must remain I/O-free. All file access goes through the `FileResolver` trait. +- Frontends should stay thin and adapt runtime-specific inputs/outputs into the shared `expand` API. +- System includes (`#include <...>`) are preserved; local includes are expanded. +- Compression must remain syntax-aware and should not be replaced with regex-based minification. +- Object-like macro spacing must be preserved during compression; shared compressor helpers should be preferred over duplicated emission logic. +- **Forbidden**: direct `std::fs`/`std::io` use inside `texpand-core`. + +## File Organization + +| What | Where | Notes | +|------|-------|-------| +| Source code | `texpand-core/src`, `texpand-cli/src`, `texpand-vscode/src`, `texpand-vscode/extension/src` | Organized by crate/frontend responsibility | +| Tests | `texpand-core/tests` | Integration-style coverage around the expansion engine | +| Configuration | repo root, `.github/workflows/`, `texpand-vscode/extension/` | Cargo and npm metadata live next to their package roots | +| Documentation | `README.md`, `.claude/CLAUDE.md`, `.project-tracker/` | README is user-facing; tracker docs are maintenance-facing | + +## Import / Module Conventions + +- **Import style**: Rust uses explicit module paths; TypeScript uses relative imports between extension files. +- **Module visibility**: public exports are intentionally small in `texpand-core`; helper state stays private to module files. +- **Circular dependencies**: code-level circular module dependencies are avoided; circular C/C++ include graphs are treated as runtime errors by the product. + +## Error Handling + +- **Error representation**: `anyhow::Result` is the common error type across Rust crates. +- **Error propagation**: use `?` and add `.context(...)` where the failing operation needs file/path context. +- **Error context**: missing files, parse failures, config failures, and clipboard problems should retain actionable context. +- **Panics / asserts**: acceptable in tests; production paths should return errors instead. + +## Testing Conventions + +- **Test location**: core behavior is covered in separate files under `texpand-core/tests`. +- **Test naming**: `test_.rs` or descriptive snake_case test functions. +- **Coverage target**: no repository-enforced threshold, but change work should maintain broad coverage of parser/expander/compressor behavior. +- **Mocking strategy**: prefer `FixtureResolver` and in-memory source maps rather than external filesystem fixtures for unit/integration tests. + +## Documentation Conventions + +- **Doc comments**: present on key public APIs and module-level responsibilities in Rust. +- **README**: acts as the primary end-user guide for CLI and extension usage. +- **CHANGELOG**: not present; recent intent is inferred from git history and tracker docs. + +## Agent Instructions + +- Use the Rust workspace as the source of truth; do not bypass `texpand-core` invariants in frontend changes. +- Prefer `cargo build --workspace`, `cargo test --workspace`, `cargo fmt --all`, and `cargo clippy --workspace -- -D warnings` for verification. +- When touching extension code, also run `npm run typecheck` from `texpand-vscode/extension/`. +- Keep changes small, preserve monorepo boundaries, and document any new runtime/config assumptions. diff --git a/.project-tracker/data-model.md b/.project-tracker/data-model.md new file mode 100644 index 0000000..71bb73e --- /dev/null +++ b/.project-tracker/data-model.md @@ -0,0 +1,44 @@ +--- +sources: + - "Cargo.toml" + - "texpand-core/src/*.rs" + - "texpand-cli/src/*.rs" + - "texpand-vscode/src/*.rs" +--- + +# Data Model + +N/A — the project has no persistent application data model, database schema, or migration layer. + +## Entities + +| Entity | Key fields | Notes | +|--------|-----------|-------| +| `ExpandOptions` | `compress` | Runtime option passed into expansion requests | +| `PreprocContext` | stack of `PreprocDirective` | Internal structural key used to decide deduplication scope | +| `TexpandConfig` | `include_paths`, `default_compress` | User config loaded by the CLI only | +| `ExpandResult` | `success`, `data`, `error` | JSON envelope emitted by the WASI frontend | +| `CompressorState` | `output`, `prev_last`, `compound_depth`, `body_nl_counter` | Internal state machine that now drives shared leaf emission behavior for compressed output | + +## Relationships + +| From | To | Cardinality | Description | +|------|----|------------|-------------| +| `ExpandState` | cached parse tree | 1:N | One expansion run caches many parsed files | +| `PreprocContext` | expanded include file | 1:N | A single conditional context may expand multiple headers | +| `TexpandConfig` | CLI invocation | 1:1 | Config is loaded once per command run and overridden by flags | + +## Schema Migrations + +| Aspect | Detail | +|--------|--------| +| Tool | N/A | +| Location | N/A | +| Strategy | No persistent schema exists | + +## Caching + +| Cache | Strategy | TTL | Invalidation | +|-------|---------|-----|-------------| +| Tree-sitter tree cache | In-memory per-run | Lifetime of one expansion call | Dropped after the process/request finishes | +| Compiled WASM module | In-memory inside extension host | Extension host lifetime | Rebuilt when extension process reloads | diff --git a/.project-tracker/deployment.md b/.project-tracker/deployment.md new file mode 100644 index 0000000..2f25181 --- /dev/null +++ b/.project-tracker/deployment.md @@ -0,0 +1,54 @@ +--- +sources: + - ".github/workflows/release.yml" + - "Cargo.toml" + - "texpand-cli/Cargo.toml" + - "texpand-vscode/Cargo.toml" + - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/package-lock.json" +--- + +# Deployment + +## Build Artifacts + +| Artifact | Format | How to build | +|----------|--------|-------------| +| `texpand-cli` | Native executable | `cargo build -p texpand-cli --release --target ` | +| `texpand-vscode.wasm` | WASI/WASM module | `cargo build -p texpand-vscode --target wasm32-wasip1 --release` | +| `texpand-vscode-.vsix` | VS Code extension package | `npm run vscode:prepublish && npm run package` in `texpand-vscode/extension/` | + +## Packaging + +- CLI packaging is target-specific binary output renamed to `texpand-`. +- Extension packaging depends on the WASM artifact plus bundled TypeScript output and is wrapped into a `.vsix`. +- The current in-repo manifest/package version target is `0.2.2`. +- Release automation uploads all artifacts to a tagged GitHub Release. + +## Environments + +| Environment | URL / target | Promotion from | Notes | +|------------|-------------|---------------|-------| +| Local development | developer machine | -- | Run Cargo and npm commands directly | +| GitHub Actions release build | GitHub-hosted runners | local/tagged source | Produces distributable binaries and VSIX | +| GitHub Release | repository releases page | release workflow | Distribution target for downloadable artifacts | + +## Health Checks + +| Check | Endpoint / command | Expected | +|-------|-------------------|----------| +| Rust workspace validation | `cargo check --workspace` | Exit code 0 | +| Test suite | `cargo test --workspace` | Exit code 0 | +| Extension typecheck | `npm run typecheck` | Exit code 0 | +| Packaging smoke check | `npm run package` | `.vsix` created successfully | + +## Monitoring & Alerts + +- No runtime production service exists, so monitoring is limited to CI/build failures and release workflow status. + +## Rollback Procedure + +1. Remove or supersede the problematic GitHub Release/tag. +2. Rebuild from the last known good commit/tag. +3. Re-upload corrected CLI binaries and VSIX artifact. +4. If the issue is extension-specific, publish or distribute a replacement VSIX built from the fixed tag. diff --git a/.project-tracker/implementation.md b/.project-tracker/implementation.md new file mode 100644 index 0000000..4bdeaf6 --- /dev/null +++ b/.project-tracker/implementation.md @@ -0,0 +1,56 @@ +--- +sources: + - "texpand-core/src/*.rs" + - "texpand-core/tests/*.rs" + - "texpand-cli/src/*.rs" + - "texpand-vscode/src/*.rs" + - "texpand-vscode/extension/src/*.ts" + - "README.md" +--- + +# Implementation Details + +## Entry Points + +| Target | File | Purpose | +|--------|------|---------| +| Core library | `texpand-core/src/lib.rs` | Re-exports parser, expander, resolver, and compressor modules | +| CLI binary | `texpand-cli/src/main.rs` | Parses flags, loads config, resolves files, and chooses stdout/file/clipboard output | +| CLI config | `texpand-cli/src/config.rs` | Loads `texpand.toml` from explicit path or XDG/home defaults | +| WASI binary | `texpand-vscode/src/main.rs` | Reads `TEXPAND_*` env vars, invokes `expand`, and prints JSON | +| VS Code extension | `texpand-vscode/extension/src/extension.ts` | Registers commands and coordinates user-facing output handling | +| VS Code WASI bridge | `texpand-vscode/extension/src/wasm.ts` | Compiles/caches WASM, mounts paths, runs the WASI process | + +## Key Algorithms & Logic + +- `parser.rs` uses tree-sitter to classify includes as local or system headers, which lets the expander inline only the local subset. +- `expander.rs` recursively walks the syntax tree, tracks the current preprocessor conditional stack, and deduplicates expansions by `(resolved_path, PreprocContext)`. +- Context normalization is token-based rather than whitespace-based, so logically identical conditions like `#if A == B` and `#if A==B` are treated the same. +- Circular includes are detected via an `expanding` set representing the active recursion stack. +- `compressor.rs` walks AST leaves and removes comments/extra whitespace while preserving token boundaries, preprocessor newlines, and macro spacing. +- Standalone compression and expansion-time compression now share the same leaf-emission helper so object-like macros such as `#define LC (i << 1)` cannot be rewritten into function-like macros. +- The VS Code wrapper converts host paths into WASI-visible paths and mounts absolute include paths only when they exist. + +## Error Handling Strategy + +- Rust crates use `anyhow::Result` with contextual messages for file access, parsing, config loading, and clipboard operations. +- CLI main returns `Result<()>`, letting Cargo print concise failures without custom panic handling. +- The WASI frontend serializes success/error into JSON so the extension can present user-facing messages. +- The extension special-cases circular dependency errors to surface a more specific message in VS Code. + +## Testing Strategy + +| Test level | Location | What it covers | +|-----------|---------|---------------| +| Unit | `texpand-core/src/parser.rs` test module and similar co-located tests | Include classification and low-level parser behavior | +| Integration | `texpand-core/tests/*.rs` | Deep expansion chains, deduplication, circular detection, compression, conditionals, system includes | +| E2E | `fixtures/` plus CLI/manual runs | Representative filesystem trees for real-world expansion scenarios | + +Recent regression coverage now explicitly checks both object-like and function-like macro spacing on the compressed expansion path. + +## Performance Considerations + +- Parse trees are cached for the duration of a single expansion run. +- Compression reuses a dedicated state machine instead of reparsing transformed output. +- The extension caches the compiled WASM module between invocations. +- No async/concurrency layer is needed; the workload is dominated by local file reads and AST traversal. diff --git a/.project-tracker/modules/cli.md b/.project-tracker/modules/cli.md new file mode 100644 index 0000000..1402f70 --- /dev/null +++ b/.project-tracker/modules/cli.md @@ -0,0 +1,33 @@ +--- +sources: + - "texpand-cli/Cargo.toml" + - "texpand-cli/src/*.rs" + - "README.md" +--- + +# Module: texpand-cli + +## Responsibility + +Native command-line frontend for expanding files from the local filesystem, reading optional user config, and returning output through stdout, a file, or the clipboard. + +This frontend now benefits from the shared compressed-emission fix because the `-c` path goes through `texpand-core`'s expansion-time compressor. + +## Entry Points + +- `texpand-cli/src/main.rs` +- `texpand-cli/src/config.rs` + +## Key Dependencies + +- `texpand-core` +- `clap` +- `toml` +- `arboard` +- `nix` on Linux for clipboard persistence via fork + +## Notable Patterns + +- CLI flags override config values where both are present. +- `-` is supported as stdin input, with the current directory used as the include base. +- Linux clipboard writes fork a child process so clipboard ownership survives parent exit. diff --git a/.project-tracker/modules/core.md b/.project-tracker/modules/core.md new file mode 100644 index 0000000..609eba3 --- /dev/null +++ b/.project-tracker/modules/core.md @@ -0,0 +1,34 @@ +--- +sources: + - "texpand-core/Cargo.toml" + - "texpand-core/src/*.rs" + - "texpand-core/tests/*.rs" +--- + +# Module: texpand-core + +## Responsibility + +Shared expansion engine for all frontends. It parses C/C++ code, resolves local includes through an abstract resolver, preserves system includes, and optionally compresses the final output. + +## Entry Points + +- `texpand-core/src/lib.rs` +- `texpand-core/src/expander.rs` +- `texpand-core/src/parser.rs` +- `texpand-core/src/compressor.rs` +- `texpand-core/src/resolver.rs` + +## Key Dependencies + +- `tree-sitter` +- `tree-sitter-cpp` +- `anyhow` +- `serde` + +## Notable Patterns + +- No direct filesystem I/O. +- Expansion deduplication is keyed by preprocessing context. +- Shared compressor leaf emission avoids semantic drift between standalone compression and compressed expansion. +- Integration tests use `FixtureResolver` to keep scenarios deterministic and fast. diff --git a/.project-tracker/modules/vscode.md b/.project-tracker/modules/vscode.md new file mode 100644 index 0000000..fdfdee2 --- /dev/null +++ b/.project-tracker/modules/vscode.md @@ -0,0 +1,38 @@ +--- +sources: + - "texpand-vscode/Cargo.toml" + - "texpand-vscode/src/*.rs" + - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/README*.md" + - "texpand-vscode/extension/l10n/*.json" + - "texpand-vscode/extension/package.nls*.json" + - "texpand-vscode/extension/src/*.ts" +--- + +# Module: texpand-vscode + +## Responsibility + +VS Code frontend composed of a Rust WASI process and a TypeScript extension host wrapper. It runs the same core expansion logic inside editor workflows and returns results to the clipboard or a sibling file. + +The Rust crate and extension manifest are now aligned on version `0.2.2`, and this module owns the extension-localized strings and README surface. + +## Entry Points + +- `texpand-vscode/src/main.rs` +- `texpand-vscode/extension/src/extension.ts` +- `texpand-vscode/extension/src/wasm.ts` + +## Key Dependencies + +- `texpand-core` +- `serde_json` +- `@vscode/wasm-wasi` +- `esbuild` +- `@vscode/vsce` + +## Notable Patterns + +- The Rust side communicates with the extension through a JSON stdout envelope. +- Host paths are translated into WASI-visible mount points before process launch. +- The extension caches the compiled WebAssembly module and dynamically mounts absolute include paths. diff --git a/.project-tracker/progress.md b/.project-tracker/progress.md new file mode 100644 index 0000000..f758f03 --- /dev/null +++ b/.project-tracker/progress.md @@ -0,0 +1,44 @@ +# Progress + +## Current Phase + +Preparing the `0.2.2` maintenance release with compressor correctness fixes and aligned package metadata. + +## Completed + +- [x] Workspace split into `texpand-core`, `texpand-cli`, and `texpand-vscode` +- [x] Tree-sitter-based C/C++ parsing and include classification +- [x] Recursive expansion with circular dependency detection +- [x] Token-aware compression that preserves preprocessor semantics +- [x] VS Code extension commands for clipboard and new-file output +- [x] GitHub Actions CI for check/fmt/clippy/test/typecheck +- [x] Tagged release workflow for native binaries and `.vsix` artifacts +- [x] Localization support and `saveBeforeExpansion` extension setting +- [x] Shared compressor leaf-emission path to preserve object-like macro spacing during compressed expansion +- [x] Version manifests aligned on `0.2.2` across workspace crates and the VS Code extension package + +## Known Issues / Technical Debt + +- [ ] No automated coverage reporting or coverage gate despite strong test coverage intent +- [ ] No dedicated lint step for the TypeScript extension in CI beyond `tsc` +- [ ] Tracker ownership remains intentionally selective for local editor settings and user-specific harness files + +## Recent Work + +- local: fixed compressed macro spacing so object-like macros no longer become function-like during expansion +- local: added regression tests for object-like and function-like macro spacing on the compressed path +- local: bumped workspace and extension manifest versions to `0.2.2` +- `be1ebb2` merged PR #7 into `main` +- `d0fff49` fixed formatting to satisfy CI +- `28063d3` updated project tracker materials +- `82fbc62` fixed compression handling for user-defined literals +- `100872e` corrected `zh-cn` localization casing +- `65b49e1` added localization support +- `72d230f` added `saveBeforeExpansion` configuration + +## Next Steps + +- Add explicit coverage tooling/reporting for the Rust core. +- Consider extension-side lint enforcement in CI. +- Evaluate end-to-end tests for the VS Code wrapper around the WASI process. +- Cut and validate the `0.2.2` release artifacts once remaining lockfile/versioning workflow decisions are settled. diff --git a/.project-tracker/stack.md b/.project-tracker/stack.md new file mode 100644 index 0000000..f4b98b5 --- /dev/null +++ b/.project-tracker/stack.md @@ -0,0 +1,51 @@ +--- +sources: + - "Cargo.toml" + - "Cargo.lock" + - "texpand-core/Cargo.toml" + - "texpand-cli/Cargo.toml" + - "texpand-vscode/Cargo.toml" + - "texpand-vscode/extension/package.json" + - ".github/workflows/*.yml" +--- + +# Technology Stack + +## Language & Runtime + +| Property | Value | +|----------|-------| +| Language | Rust 2024 edition, TypeScript | +| Runtime / VM | Native binaries for CLI, WASI/WASM for VS Code core, Node.js 22 in CI for extension tooling | +| Package manager | Cargo workspace + npm (`package-lock.json`) | +| Release version | `0.2.2` across workspace crates and VS Code package metadata | + +## Frameworks & Libraries + +| Dependency | Version | Purpose | +|-----------|---------|---------| +| `tree-sitter` | `0.24` | Parse C/C++ source into ASTs instead of using regex/string heuristics. | +| `tree-sitter-cpp` | `0.23` | C/C++ grammar backing include extraction and token-aware traversal. | +| `anyhow` | `1` | Context-rich error propagation across CLI and WASM boundaries. | +| `serde` / `serde_json` | `1` | Config parsing and JSON result exchange for the WASM process. | +| `clap` | `4` | Structured CLI argument parsing and help generation. | +| `toml` | `1` | Read user config from `texpand.toml`. | +| `arboard` | `3` | Clipboard support for the CLI output path. | +| `@vscode/wasm-wasi` | `^1.0.0` | Runs the Rust frontend as a WASI process inside VS Code. | +| `esbuild` | `^0.28.0` | Bundles the extension TypeScript entry point. | +| `@vscode/vsce` | `^3.9.1` | Packages the extension as a VSIX for release. | + +## Database & Storage + +| Component | Technology | Purpose | +|-----------|-----------|---------| +| Primary DB | N/A | This project has no database layer. | +| ORM / Client | N/A | No persistent relational or document store. | +| Cache | In-memory parse tree cache in `expander.rs` | Avoid reparsing the same include graph within one expansion run. | +| File storage | Local filesystem / VS Code workspace FS | Reads source/header files and writes optional CLI / extension outputs. | + +## Infrastructure & Services + +- GitHub Actions -- workspace CI, cross-platform binary builds, and tagged release packaging. +- GitHub Releases -- distribution channel for CLI binaries and the VS Code `.vsix`. +- VS Code Marketplace-compatible packaging -- extension bundle output via `vsce`. diff --git a/.project-tracker/toolchain.md b/.project-tracker/toolchain.md new file mode 100644 index 0000000..b6cf35a --- /dev/null +++ b/.project-tracker/toolchain.md @@ -0,0 +1,58 @@ +--- +sources: + - ".github/workflows/*.yml" + - ".cargo/config.toml" + - "Cargo.toml" + - "Cargo.lock" + - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/tsconfig.json" + - "texpand-vscode/extension/eslint.config.mjs" + - ".claude/CLAUDE.md" +--- + +# Toolchain & Dev Setup + +## Build System + +| Tool | Command | Output | +|------|---------|--------| +| Cargo workspace | `cargo build --workspace` | All Rust crates | +| Cargo release build | `cargo build -p texpand-cli --release --target ` | Native CLI binary per target | +| Cargo WASI build | `cargo build -p texpand-vscode --target wasm32-wasip1 --release` | `texpand-vscode.wasm` | +| npm scripts | `npm run vscode:prepublish` | Optimized WASM + bundled extension JS | +| VSCE | `npm run package` | `.vsix` extension artifact | + +Current package metadata is aligned for the next `0.2.2` release cut across the Cargo workspace and the VS Code extension manifest. + +## Linting & Formatting + +| Tool | Config file | Run command | +|------|-----------|-------------| +| `rustfmt` | default config | `cargo fmt --all -- --check` | +| `clippy` | workspace defaults | `cargo clippy --workspace -- -D warnings` | +| TypeScript compiler | `texpand-vscode/extension/tsconfig.json` | `npm run typecheck` | +| ESLint | `texpand-vscode/extension/eslint.config.mjs` | ad hoc/local use; CI currently relies on `tsc` rather than a dedicated lint step | + +## Testing + +| Aspect | Detail | +|--------|--------| +| Framework | Rust built-in test harness | +| Coverage target | No automated coverage gate in repo; ECC instructions require 80%+ for changes | +| Coverage tool | Not configured | +| E2E / integration | `texpand-core/tests/` uses an in-memory `FixtureResolver`; `fixtures/` supports CLI-style scenarios | + +## CI/CD Pipeline + +- `ci.yml` runs `cargo check`, `cargo fmt --check`, `cargo clippy -D warnings`, `cargo test --workspace`, and extension `npm run typecheck`. +- `release.yml` triggers on tags matching `v*`. +- Release jobs build CLI binaries for Linux, macOS, and Windows, then build/package the VS Code extension on Ubuntu. +- Tagged artifacts are uploaded and attached to a GitHub Release via `softprops/action-gh-release`. + +## Development Environment + +| Requirement | Value | +|-----------|-------| +| Required tools | Rust stable toolchain, Cargo, Node.js 22 for extension work, npm, `wasm32-wasip1` target | +| Environment variables | `HOME` or `XDG_CONFIG_HOME` for CLI config discovery; `TEXPAND_*` variables are set by the VS Code wrapper at runtime | +| Dev server / watcher | `npm run watch` from `texpand-vscode/extension/` for extension bundling | diff --git a/Cargo.lock b/Cargo.lock index afeb2bd..4211925 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -718,7 +718,7 @@ dependencies = [ [[package]] name = "texpand-cli" -version = "0.2.1" +version = "0.2.2" dependencies = [ "anyhow", "arboard", @@ -731,7 +731,7 @@ dependencies = [ [[package]] name = "texpand-core" -version = "0.2.1" +version = "0.2.2" dependencies = [ "anyhow", "serde", @@ -741,7 +741,7 @@ dependencies = [ [[package]] name = "texpand-vscode" -version = "0.1.2" +version = "0.2.2" dependencies = [ "anyhow", "serde", diff --git a/texpand-cli/Cargo.toml b/texpand-cli/Cargo.toml index 29066b9..9b7cfad 100644 --- a/texpand-cli/Cargo.toml +++ b/texpand-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "texpand-cli" -version = "0.2.1" +version = "0.2.2" edition = "2024" [dependencies] diff --git a/texpand-core/Cargo.toml b/texpand-core/Cargo.toml index 5fd9e32..7061a75 100644 --- a/texpand-core/Cargo.toml +++ b/texpand-core/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "texpand-core" -version = "0.2.1" +version = "0.2.2" edition = "2024" [dependencies] diff --git a/texpand-core/src/compressor.rs b/texpand-core/src/compressor.rs index 2dc59db..00e39d6 100644 --- a/texpand-core/src/compressor.rs +++ b/texpand-core/src/compressor.rs @@ -124,6 +124,32 @@ impl CompressorState { } } +pub(crate) fn emit_leaf_token( + st: &mut CompressorState, + node: &tree_sitter::Node, + cursor: &tree_sitter::TreeCursor, + source: &str, +) { + if node.kind() == "comment" { + return; + } + + if let Ok(text) = node.utf8_text(source.as_bytes()) + && !text.is_empty() + { + st.emit_token(text, node.kind() == "literal_suffix"); + + // `#define FOO …` needs a separating space after the macro name so + // object-like macros do not become function-like macros. + if cursor.field_name() == Some("name") + && let Some(parent) = node.parent() + && parent.kind() == "preproc_def" + { + st.ensure_trailing_space(); + } + } +} + // ── Public API ────────────────────────────────────────────────────────────── /// Compress C/C++ source code while preserving semantic correctness. @@ -208,22 +234,8 @@ fn compress_impl( let node = cursor.node(); let is_leaf = node.child_count() == 0; - if is_leaf - && node.kind() != "comment" - && let Ok(text) = node.utf8_text(source.as_bytes()) - && !text.is_empty() - { - st.emit_token(text, node.kind() == "literal_suffix"); - // `#define FOO …` — ensure at least one space between the - // macro name and the replacement text so that - // #define FOO"abc" and #define FOO(expr) - // don't become syntactically wrong. - if cursor.field_name() == Some("name") - && let Some(parent) = node.parent() - && parent.kind() == "preproc_def" - { - st.ensure_trailing_space(); - } + if is_leaf { + emit_leaf_token(&mut st, &node, &cursor, source); } if !skip_node(&node, source) && cursor.goto_first_child() { diff --git a/texpand-core/src/expander.rs b/texpand-core/src/expander.rs index fd06fa0..89d1697 100644 --- a/texpand-core/src/expander.rs +++ b/texpand-core/src/expander.rs @@ -285,10 +285,8 @@ fn expand_recursive( if node.child_count() == 0 && let Some(ref mut cs) = cs { - if node.kind() != "comment" - && let Ok(text) = node.utf8_text(source.as_bytes()) - { - cs.emit_token(text, node.kind() == "literal_suffix"); + if node.kind() != "comment" { + compressor::emit_leaf_token(cs, &node, &cursor, source); } if node_start > byte_pos { output.push_str(&source[byte_pos..node_start]); @@ -532,4 +530,32 @@ mod tests { assert!(result.contains("#ifdef USE_FOO\nint a=1;")); assert!(result.contains("int x;")); } + + #[test] + fn test_compressed_object_like_macro_keeps_space() { + let src = "#define LC (i << 1)\nint x = LC;\n"; + let result = expand_compressed("main.cpp", src, &MockResolver).unwrap(); + assert!( + result.contains("#define LC ("), + "object-like macro spacing broken: {result:?}" + ); + assert!( + !result.contains("#define LC("), + "object-like macro became function-like: {result:?}" + ); + } + + #[test] + fn test_compressed_function_like_macro_keeps_no_space() { + let src = "#define LC(i) ((i) << 1)\nint x = LC(2);\n"; + let result = expand_compressed("main.cpp", src, &MockResolver).unwrap(); + assert!( + result.contains("#define LC("), + "function-like macro formatting broken: {result:?}" + ); + assert!( + !result.contains("#define LC (i)"), + "function-like macro got false object-like spacing: {result:?}" + ); + } } diff --git a/texpand-vscode/Cargo.toml b/texpand-vscode/Cargo.toml index dfc3b70..05fa04e 100644 --- a/texpand-vscode/Cargo.toml +++ b/texpand-vscode/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "texpand-vscode" -version = "0.1.2" +version = "0.2.2" edition = "2024" [dependencies] diff --git a/texpand-vscode/extension/package.json b/texpand-vscode/extension/package.json index 77bca9b..55521a5 100644 --- a/texpand-vscode/extension/package.json +++ b/texpand-vscode/extension/package.json @@ -3,7 +3,7 @@ "displayName": "%extension.displayName%", "description": "%extension.description%", "l10n": "./l10n", - "version": "0.2.1", + "version": "0.2.2", "publisher": "explodingkonjac", "repository": { "type": "git", From f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698 Mon Sep 17 00:00:00 2001 From: ExplodingKonjac Date: Thu, 4 Jun 2026 16:57:48 +0800 Subject: [PATCH 2/3] chore: update project state --- .../project-tracker}/.state.json | 78 ++++++++++++------- .../project-tracker}/INDEX.md | 3 +- .../project-tracker}/api.md | 0 .../project-tracker}/architecture.md | 0 .../project-tracker}/conventions.md | 2 +- .../project-tracker}/data-model.md | 0 .../project-tracker}/deployment.md | 2 +- .../project-tracker}/implementation.md | 4 +- .../project-tracker}/modules/cli.md | 0 .../project-tracker}/modules/core.md | 2 +- .../project-tracker}/modules/vscode.md | 2 +- .../project-tracker}/progress.md | 11 +-- .../project-tracker}/stack.md | 3 +- .../project-tracker}/toolchain.md | 3 +- texpand-vscode/extension/package-lock.json | 4 +- 15 files changed, 72 insertions(+), 42 deletions(-) rename {.project-tracker => .agents/project-tracker}/.state.json (69%) rename {.project-tracker => .agents/project-tracker}/INDEX.md (94%) rename {.project-tracker => .agents/project-tracker}/api.md (100%) rename {.project-tracker => .agents/project-tracker}/architecture.md (100%) rename {.project-tracker => .agents/project-tracker}/conventions.md (97%) rename {.project-tracker => .agents/project-tracker}/data-model.md (100%) rename {.project-tracker => .agents/project-tracker}/deployment.md (97%) rename {.project-tracker => .agents/project-tracker}/implementation.md (90%) rename {.project-tracker => .agents/project-tracker}/modules/cli.md (100%) rename {.project-tracker => .agents/project-tracker}/modules/core.md (87%) rename {.project-tracker => .agents/project-tracker}/modules/vscode.md (86%) rename {.project-tracker => .agents/project-tracker}/progress.md (75%) rename {.project-tracker => .agents/project-tracker}/stack.md (93%) rename {.project-tracker => .agents/project-tracker}/toolchain.md (94%) diff --git a/.project-tracker/.state.json b/.agents/project-tracker/.state.json similarity index 69% rename from .project-tracker/.state.json rename to .agents/project-tracker/.state.json index 1a518f0..a10f0f4 100644 --- a/.project-tracker/.state.json +++ b/.agents/project-tracker/.state.json @@ -1,7 +1,10 @@ { "files": { "INDEX.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": { + "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + }, "matched_paths": [ ".cargo/config.toml", ".github/workflows/ci.yml", @@ -12,12 +15,14 @@ "texpand-cli/Cargo.toml", "texpand-core/Cargo.toml", "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/package-lock.json", "texpand-vscode/extension/package.json" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "api.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "README.md", "texpand-cli/src/config.rs", @@ -27,10 +32,11 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "architecture.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ ".claude/CLAUDE.md", "Cargo.toml", @@ -45,10 +51,11 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "conventions.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ ".claude/CLAUDE.md", "Cargo.toml", @@ -65,10 +72,11 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/extension/tsconfig.json" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "data-model.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "Cargo.toml", "texpand-cli/src/config.rs", @@ -80,10 +88,13 @@ "texpand-core/src/resolver.rs", "texpand-vscode/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "deployment.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": { + "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + }, "matched_paths": [ ".github/workflows/release.yml", "Cargo.toml", @@ -92,10 +103,11 @@ "texpand-vscode/extension/package-lock.json", "texpand-vscode/extension/package.json" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "implementation.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "README.md", "texpand-cli/src/config.rs", @@ -116,20 +128,22 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "modules/cli.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "README.md", "texpand-cli/Cargo.toml", "texpand-cli/src/config.rs", "texpand-cli/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "modules/core.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "texpand-core/Cargo.toml", "texpand-core/src/compressor.rs", @@ -145,10 +159,11 @@ "texpand-core/tests/test_edge_cases.rs", "texpand-core/tests/test_system_include.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "modules/vscode.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": {}, "matched_paths": [ "texpand-vscode/Cargo.toml", "texpand-vscode/extension/README.md", @@ -163,15 +178,21 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "progress.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": { + "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + }, "matched_paths": [], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "stack.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": { + "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + }, "matched_paths": [ ".github/workflows/ci.yml", ".github/workflows/release.yml", @@ -180,12 +201,16 @@ "texpand-cli/Cargo.toml", "texpand-core/Cargo.toml", "texpand-vscode/Cargo.toml", + "texpand-vscode/extension/package-lock.json", "texpand-vscode/extension/package.json" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" }, "toolchain.md": { - "baseline": "be1ebb2d831725164240d1761ee6f07ad989ff8f", + "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "changed_fingerprints": { + "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + }, "matched_paths": [ ".cargo/config.toml", ".claude/CLAUDE.md", @@ -194,10 +219,11 @@ "Cargo.lock", "Cargo.toml", "texpand-vscode/extension/eslint.config.mjs", + "texpand-vscode/extension/package-lock.json", "texpand-vscode/extension/package.json", "texpand-vscode/extension/tsconfig.json" ], - "updated": "2026-05-29T14:24:46Z" + "updated": "2026-06-04T08:56:57Z" } }, "version": 1 diff --git a/.project-tracker/INDEX.md b/.agents/project-tracker/INDEX.md similarity index 94% rename from .project-tracker/INDEX.md rename to .agents/project-tracker/INDEX.md index fd32db5..0955956 100644 --- a/.project-tracker/INDEX.md +++ b/.agents/project-tracker/INDEX.md @@ -7,6 +7,7 @@ sources: - ".cargo/config.toml" - "texpand-*/Cargo.toml" - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/package-lock.json" --- # PROJECT: Template-Expand @@ -39,7 +40,7 @@ Rust monorepo for expanding local C/C++ template headers into a single-file subm - `texpand-core` is intentionally I/O-free and exposes the reusable expansion engine. - `texpand-cli` provides local filesystem, config, and clipboard integration. - `texpand-vscode` compiles the Rust frontend to WASI/WASM and wraps it in a VS Code extension. -- Workspace crate manifests and extension package metadata are currently aligned on release version `0.2.2`. +- Workspace crate manifests, extension package metadata, and extension lockfile are currently aligned on release version `0.2.2`. - Releases publish native CLI binaries and a `.vsix` extension artifact. ## Quick Reference Commands diff --git a/.project-tracker/api.md b/.agents/project-tracker/api.md similarity index 100% rename from .project-tracker/api.md rename to .agents/project-tracker/api.md diff --git a/.project-tracker/architecture.md b/.agents/project-tracker/architecture.md similarity index 100% rename from .project-tracker/architecture.md rename to .agents/project-tracker/architecture.md diff --git a/.project-tracker/conventions.md b/.agents/project-tracker/conventions.md similarity index 97% rename from .project-tracker/conventions.md rename to .agents/project-tracker/conventions.md index 3bfcd19..32ec3db 100644 --- a/.project-tracker/conventions.md +++ b/.agents/project-tracker/conventions.md @@ -53,7 +53,7 @@ sources: | Source code | `texpand-core/src`, `texpand-cli/src`, `texpand-vscode/src`, `texpand-vscode/extension/src` | Organized by crate/frontend responsibility | | Tests | `texpand-core/tests` | Integration-style coverage around the expansion engine | | Configuration | repo root, `.github/workflows/`, `texpand-vscode/extension/` | Cargo and npm metadata live next to their package roots | -| Documentation | `README.md`, `.claude/CLAUDE.md`, `.project-tracker/` | README is user-facing; tracker docs are maintenance-facing | +| Documentation | `README.md`, `.claude/CLAUDE.md`, `.agents/project-tracker/` | README is user-facing; tracker docs are maintenance-facing | ## Import / Module Conventions diff --git a/.project-tracker/data-model.md b/.agents/project-tracker/data-model.md similarity index 100% rename from .project-tracker/data-model.md rename to .agents/project-tracker/data-model.md diff --git a/.project-tracker/deployment.md b/.agents/project-tracker/deployment.md similarity index 97% rename from .project-tracker/deployment.md rename to .agents/project-tracker/deployment.md index 2f25181..7b100fa 100644 --- a/.project-tracker/deployment.md +++ b/.agents/project-tracker/deployment.md @@ -22,7 +22,7 @@ sources: - CLI packaging is target-specific binary output renamed to `texpand-`. - Extension packaging depends on the WASM artifact plus bundled TypeScript output and is wrapped into a `.vsix`. -- The current in-repo manifest/package version target is `0.2.2`. +- The current in-repo manifest/package version target is `0.2.2`, including the extension package lockfile root package entry. - Release automation uploads all artifacts to a tagged GitHub Release. ## Environments diff --git a/.project-tracker/implementation.md b/.agents/project-tracker/implementation.md similarity index 90% rename from .project-tracker/implementation.md rename to .agents/project-tracker/implementation.md index 4bdeaf6..2503a69 100644 --- a/.project-tracker/implementation.md +++ b/.agents/project-tracker/implementation.md @@ -28,7 +28,7 @@ sources: - Context normalization is token-based rather than whitespace-based, so logically identical conditions like `#if A == B` and `#if A==B` are treated the same. - Circular includes are detected via an `expanding` set representing the active recursion stack. - `compressor.rs` walks AST leaves and removes comments/extra whitespace while preserving token boundaries, preprocessor newlines, and macro spacing. -- Standalone compression and expansion-time compression now share the same leaf-emission helper so object-like macros such as `#define LC (i << 1)` cannot be rewritten into function-like macros. +- Standalone compression and expansion-time compression now share the same leaf-emission helper. That helper preserves user-defined literal suffixes and keeps object-like macros such as `#define LC (i << 1)` from being rewritten into function-like macros. - The VS Code wrapper converts host paths into WASI-visible paths and mounts absolute include paths only when they exist. ## Error Handling Strategy @@ -46,7 +46,7 @@ sources: | Integration | `texpand-core/tests/*.rs` | Deep expansion chains, deduplication, circular detection, compression, conditionals, system includes | | E2E | `fixtures/` plus CLI/manual runs | Representative filesystem trees for real-world expansion scenarios | -Recent regression coverage now explicitly checks both object-like and function-like macro spacing on the compressed expansion path. +Recent regression coverage explicitly checks user-defined literal suffixes plus object-like and function-like macro spacing on the compressed expansion path. ## Performance Considerations diff --git a/.project-tracker/modules/cli.md b/.agents/project-tracker/modules/cli.md similarity index 100% rename from .project-tracker/modules/cli.md rename to .agents/project-tracker/modules/cli.md diff --git a/.project-tracker/modules/core.md b/.agents/project-tracker/modules/core.md similarity index 87% rename from .project-tracker/modules/core.md rename to .agents/project-tracker/modules/core.md index 609eba3..b989a6b 100644 --- a/.project-tracker/modules/core.md +++ b/.agents/project-tracker/modules/core.md @@ -30,5 +30,5 @@ Shared expansion engine for all frontends. It parses C/C++ code, resolves local - No direct filesystem I/O. - Expansion deduplication is keyed by preprocessing context. -- Shared compressor leaf emission avoids semantic drift between standalone compression and compressed expansion. +- Shared compressor leaf emission avoids semantic drift between standalone compression and compressed expansion, including user-defined literal suffixes and object-like macro spacing. - Integration tests use `FixtureResolver` to keep scenarios deterministic and fast. diff --git a/.project-tracker/modules/vscode.md b/.agents/project-tracker/modules/vscode.md similarity index 86% rename from .project-tracker/modules/vscode.md rename to .agents/project-tracker/modules/vscode.md index fdfdee2..0fa1921 100644 --- a/.project-tracker/modules/vscode.md +++ b/.agents/project-tracker/modules/vscode.md @@ -15,7 +15,7 @@ sources: VS Code frontend composed of a Rust WASI process and a TypeScript extension host wrapper. It runs the same core expansion logic inside editor workflows and returns results to the clipboard or a sibling file. -The Rust crate and extension manifest are now aligned on version `0.2.2`, and this module owns the extension-localized strings and README surface. +The Rust crate, extension manifest, and extension lockfile are now aligned on version `0.2.2`, and this module owns the extension-localized strings and README surface. ## Entry Points diff --git a/.project-tracker/progress.md b/.agents/project-tracker/progress.md similarity index 75% rename from .project-tracker/progress.md rename to .agents/project-tracker/progress.md index f758f03..d55928e 100644 --- a/.project-tracker/progress.md +++ b/.agents/project-tracker/progress.md @@ -15,7 +15,7 @@ Preparing the `0.2.2` maintenance release with compressor correctness fixes and - [x] Tagged release workflow for native binaries and `.vsix` artifacts - [x] Localization support and `saveBeforeExpansion` extension setting - [x] Shared compressor leaf-emission path to preserve object-like macro spacing during compressed expansion -- [x] Version manifests aligned on `0.2.2` across workspace crates and the VS Code extension package +- [x] Version manifests and the extension npm lockfile aligned on `0.2.2` ## Known Issues / Technical Debt @@ -25,9 +25,10 @@ Preparing the `0.2.2` maintenance release with compressor correctness fixes and ## Recent Work -- local: fixed compressed macro spacing so object-like macros no longer become function-like during expansion -- local: added regression tests for object-like and function-like macro spacing on the compressed path -- local: bumped workspace and extension manifest versions to `0.2.2` +- local: updated the extension package lockfile root package version to `0.2.2` +- `7336614` fixed compressed macro spacing so object-like macros no longer become function-like during expansion +- `7336614` added regression tests for object-like and function-like macro spacing on the compressed path +- `7336614` aligned workspace and extension package manifests on `0.2.2` - `be1ebb2` merged PR #7 into `main` - `d0fff49` fixed formatting to satisfy CI - `28063d3` updated project tracker materials @@ -41,4 +42,4 @@ Preparing the `0.2.2` maintenance release with compressor correctness fixes and - Add explicit coverage tooling/reporting for the Rust core. - Consider extension-side lint enforcement in CI. - Evaluate end-to-end tests for the VS Code wrapper around the WASI process. -- Cut and validate the `0.2.2` release artifacts once remaining lockfile/versioning workflow decisions are settled. +- Cut and validate the `0.2.2` release artifacts now that manifest and lockfile versioning are aligned. diff --git a/.project-tracker/stack.md b/.agents/project-tracker/stack.md similarity index 93% rename from .project-tracker/stack.md rename to .agents/project-tracker/stack.md index f4b98b5..ba12d7f 100644 --- a/.project-tracker/stack.md +++ b/.agents/project-tracker/stack.md @@ -6,6 +6,7 @@ sources: - "texpand-cli/Cargo.toml" - "texpand-vscode/Cargo.toml" - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/package-lock.json" - ".github/workflows/*.yml" --- @@ -18,7 +19,7 @@ sources: | Language | Rust 2024 edition, TypeScript | | Runtime / VM | Native binaries for CLI, WASI/WASM for VS Code core, Node.js 22 in CI for extension tooling | | Package manager | Cargo workspace + npm (`package-lock.json`) | -| Release version | `0.2.2` across workspace crates and VS Code package metadata | +| Release version | `0.2.2` across workspace crates, VS Code package metadata, and npm lockfile | ## Frameworks & Libraries diff --git a/.project-tracker/toolchain.md b/.agents/project-tracker/toolchain.md similarity index 94% rename from .project-tracker/toolchain.md rename to .agents/project-tracker/toolchain.md index b6cf35a..62a63c3 100644 --- a/.project-tracker/toolchain.md +++ b/.agents/project-tracker/toolchain.md @@ -5,6 +5,7 @@ sources: - "Cargo.toml" - "Cargo.lock" - "texpand-vscode/extension/package.json" + - "texpand-vscode/extension/package-lock.json" - "texpand-vscode/extension/tsconfig.json" - "texpand-vscode/extension/eslint.config.mjs" - ".claude/CLAUDE.md" @@ -22,7 +23,7 @@ sources: | npm scripts | `npm run vscode:prepublish` | Optimized WASM + bundled extension JS | | VSCE | `npm run package` | `.vsix` extension artifact | -Current package metadata is aligned for the next `0.2.2` release cut across the Cargo workspace and the VS Code extension manifest. +Current package metadata is aligned for the next `0.2.2` release cut across the Cargo workspace, the VS Code extension manifest, and the extension npm lockfile. ## Linting & Formatting diff --git a/texpand-vscode/extension/package-lock.json b/texpand-vscode/extension/package-lock.json index 6b2a6b9..60e8fdc 100644 --- a/texpand-vscode/extension/package-lock.json +++ b/texpand-vscode/extension/package-lock.json @@ -1,12 +1,12 @@ { "name": "texpand-vscode", - "version": "0.2.0", + "version": "0.2.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "texpand-vscode", - "version": "0.2.0", + "version": "0.2.2", "dependencies": { "@vscode/vsce": "^3.9.1", "@vscode/wasm-wasi": "^1.0.0" From 32f7e106deea5980acde02d346c9ad59f6841f46 Mon Sep 17 00:00:00 2001 From: ExplodingKonjac Date: Thu, 4 Jun 2026 17:02:25 +0800 Subject: [PATCH 3/3] doc: update README --- .agents/project-tracker/.state.json | 47 +++++++++++++++--------- .agents/project-tracker/progress.md | 1 + README.md | 3 +- texpand-vscode/extension/README.md | 9 ++++- texpand-vscode/extension/README.zh-CN.md | 9 ++++- 5 files changed, 48 insertions(+), 21 deletions(-) diff --git a/.agents/project-tracker/.state.json b/.agents/project-tracker/.state.json index a10f0f4..73ceec4 100644 --- a/.agents/project-tracker/.state.json +++ b/.agents/project-tracker/.state.json @@ -1,9 +1,9 @@ { "files": { "INDEX.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", "changed_fingerprints": { - "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + "README.md": "806935c899e73c6866646f5bf078ae50ecb1e8928cfacefbb555c60803da35ca" }, "matched_paths": [ ".cargo/config.toml", @@ -18,11 +18,13 @@ "texpand-vscode/extension/package-lock.json", "texpand-vscode/extension/package.json" ], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "api.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", - "changed_fingerprints": {}, + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", + "changed_fingerprints": { + "README.md": "806935c899e73c6866646f5bf078ae50ecb1e8928cfacefbb555c60803da35ca" + }, "matched_paths": [ "README.md", "texpand-cli/src/config.rs", @@ -32,7 +34,7 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "architecture.md": { "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", @@ -106,8 +108,10 @@ "updated": "2026-06-04T08:56:57Z" }, "implementation.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", - "changed_fingerprints": {}, + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", + "changed_fingerprints": { + "README.md": "806935c899e73c6866646f5bf078ae50ecb1e8928cfacefbb555c60803da35ca" + }, "matched_paths": [ "README.md", "texpand-cli/src/config.rs", @@ -128,18 +132,20 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "modules/cli.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", - "changed_fingerprints": {}, + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", + "changed_fingerprints": { + "README.md": "806935c899e73c6866646f5bf078ae50ecb1e8928cfacefbb555c60803da35ca" + }, "matched_paths": [ "README.md", "texpand-cli/Cargo.toml", "texpand-cli/src/config.rs", "texpand-cli/src/main.rs" ], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "modules/core.md": { "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", @@ -162,8 +168,11 @@ "updated": "2026-06-04T08:56:57Z" }, "modules/vscode.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", - "changed_fingerprints": {}, + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", + "changed_fingerprints": { + "texpand-vscode/extension/README.md": "3c04abc8e1a08d3e4d66130bf5fecaa1d26318fe4ff2afcb8a726a45407937d3", + "texpand-vscode/extension/README.zh-CN.md": "401a0fd9075c62d773e709b2944e60f7dedde1b5990b8a1fcbb3422084498eab" + }, "matched_paths": [ "texpand-vscode/Cargo.toml", "texpand-vscode/extension/README.md", @@ -178,15 +187,17 @@ "texpand-vscode/extension/src/wasm.ts", "texpand-vscode/src/main.rs" ], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "progress.md": { - "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", + "baseline": "f66e39cb2ad9b62b5f57bafa8ee2d18cdf86d698", "changed_fingerprints": { - "texpand-vscode/extension/package-lock.json": "e44e45de12582c728ddf212f4fc1b69d2e8afcb8ce8e7e8081094a3e788c3978" + "README.md": "806935c899e73c6866646f5bf078ae50ecb1e8928cfacefbb555c60803da35ca", + "texpand-vscode/extension/README.md": "3c04abc8e1a08d3e4d66130bf5fecaa1d26318fe4ff2afcb8a726a45407937d3", + "texpand-vscode/extension/README.zh-CN.md": "401a0fd9075c62d773e709b2944e60f7dedde1b5990b8a1fcbb3422084498eab" }, "matched_paths": [], - "updated": "2026-06-04T08:56:57Z" + "updated": "2026-06-04T08:59:49Z" }, "stack.md": { "baseline": "7336614740fff8cd54fb0763af1544da086df7e9", diff --git a/.agents/project-tracker/progress.md b/.agents/project-tracker/progress.md index d55928e..ba00b11 100644 --- a/.agents/project-tracker/progress.md +++ b/.agents/project-tracker/progress.md @@ -25,6 +25,7 @@ Preparing the `0.2.2` maintenance release with compressor correctness fixes and ## Recent Work +- local: updated the root and VS Code extension READMEs for `0.2.2`, `saveBeforeExpansion`, and compression correctness fixes - local: updated the extension package lockfile root package version to `0.2.2` - `7336614` fixed compressed macro spacing so object-like macros no longer become function-like during expansion - `7336614` added regression tests for object-like and function-like macro spacing on the compressed path diff --git a/README.md b/README.md index ac5431b..6fd482b 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Template-Expand 是一个专为 C/C++ Competitive Programming 设计的本地模 - **精确解析**：基于 Tree-sitter 增量解析生成 AST。 - **循环依赖处理**：将头文件包含关系构建为图，自动检测循环包含并报错，按正确的拓扑排序拼接代码。 -- **安全压缩**：提供可选的代码压缩功能。通过遍历 AST 叶子节点，仅去除注释和多余空格，并在关键词法边界（如标识符之间）保留必要空格，确保压缩后的代码语义不发生任何改变。 +- **安全压缩**：提供可选的代码压缩功能。通过遍历 AST 叶子节点，仅去除注释和多余空格，并在词法边界（如标识符之间）保留必要空格；同时保留用户定义字面量（如 `123_km`）和宏定义中的关键空格，避免对象宏被误压缩成函数宏。 - **跨平台双端支持**：提供本地命令行工具与 VSCode 扩展两种使用形态，两端共享同一套底层 Rust 解析逻辑。 ## 基础架构概览 @@ -97,6 +97,7 @@ texpand main.cpp --config /path/to/my-config.toml | `texpand.includePaths` | `string[]` | `["./"]` | 本地模板头文件的搜索路径。支持相对于当前工作区根目录的相对路径，或系统的绝对路径。 | | `texpand.defaultCompression` | `boolean` | `false` | 展开代码时，是否默认开启去注释、去多余空格的 Token 级安全压缩。 | | `texpand.outputMode` | `enum` | `"clipboard"` | 结果输出方式。可选值：`"clipboard"` (复制到剪贴板)、`"newFile"` (在当前目录下生成 `.expanded.cpp` 文件)。 | +| `texpand.saveBeforeExpansion` | `boolean` | `true` | 展开前是否先保存当前编辑器文件，确保 WASM 进程读取到最新内容。 | ### 注册命令 (Commands) diff --git a/texpand-vscode/extension/README.md b/texpand-vscode/extension/README.md index debe429..f705313 100644 --- a/texpand-vscode/extension/README.md +++ b/texpand-vscode/extension/README.md @@ -15,7 +15,7 @@ The extension runs the core Rust logic via WebAssembly (WASI) — no local CLI t ## Features - **One-click expansion**: Expand all local `#include` headers directly in the editor — via the editor title button, context menu, or command palette. -- **Safe code compression**: Optionally strip comments and unnecessary whitespace with semantic-aware token merging that won't break your code. +- **Safe code compression**: Optionally strip comments and unnecessary whitespace with semantic-aware token merging that preserves user-defined literals and macro spacing. - **Zero external dependency**: The WASM engine is bundled with the extension. No binary tools or Rust environment needed. - **Remote development ready**: Works seamlessly with Remote-SSH, Dev Containers, and VSCode for Web via the virtual file system. - **Flexible output**: Copy expanded code to clipboard, or write to a `.expanded.cpp` file alongside the original. @@ -38,6 +38,7 @@ Search for `texpand` in VSCode settings: | `texpand.includePaths` | `string[]` | `["./"]` | Header search paths. Supports workspace-relative or absolute paths. | | `texpand.defaultCompression` | `boolean` | `false` | Whether to enable code compression by default. | | `texpand.outputMode` | `"clipboard"` / `"newFile"` | `"clipboard"` | Output destination: clipboard, or a new `.expanded.cpp` file. | +| `texpand.saveBeforeExpansion` | `boolean` | `true` | Whether to save the active file before expansion so the WASM process reads the latest content. | ## Commands @@ -53,6 +54,12 @@ None yet. ## Release Notes +### 0.2.2 + +- Fix: compressed expansion now preserves object-like macro spacing, so macros such as `#define LC (i << 1)` do not become function-like macros. +- Fix: the standalone compressor and expansion-time compressor now share the same token emission path, including user-defined literal handling. +- Chore: align Rust crate manifests, VS Code extension manifest, and the npm lockfile on version `0.2.2`. + ### 0.2.0 - Feature: add a configuration `saveBeforeExpansion` to control whether the current file would be saved before expansion. Default value is `true`. diff --git a/texpand-vscode/extension/README.zh-CN.md b/texpand-vscode/extension/README.zh-CN.md index 512b859..c834474 100644 --- a/texpand-vscode/extension/README.zh-CN.md +++ b/texpand-vscode/extension/README.zh-CN.md @@ -15,7 +15,7 @@ ## 核心功能 - **一键展开**：通过编辑器标题栏按钮、右键菜单或命令面板，一键展开所有本地头文件依赖。 -- **安全压缩**：可选地去除注释和多余空格，同时在关键词法边界保留必要空格，确保压缩后的代码语义不变。 +- **安全压缩**：可选地去除注释和多余空格，同时在词法边界保留必要空格，并正确保留用户定义字面量和宏定义中的关键空格。 - **零外部依赖**：扩展自带 WASM 引擎，无需额外安装任何二进制工具。 - **远程开发兼容**：通过 VSCode 虚拟文件系统工作，完全兼容 Remote-SSH、Dev Containers 和 VSCode for Web。 - **灵活输出**：支持自动复制到剪贴板，或在源文件同级目录生成 `.expanded.cpp` 文件。 @@ -38,6 +38,7 @@ | `texpand.includePaths` | `string[]` | `["./"]` | 头文件搜索路径。支持工作区相对路径或系统绝对路径。 | | `texpand.defaultCompression` | `boolean` | `false` | 展开时是否默认开启代码压缩。 | | `texpand.outputMode` | `"clipboard"` / `"newFile"` | `"clipboard"` | 展开结果的输出方式：复制到剪贴板，或生成新文件。 | +| `texpand.saveBeforeExpansion` | `boolean` | `true` | 展开前是否先保存当前文件，确保 WASM 进程读取到最新内容。 | ## 命令列表 @@ -53,6 +54,12 @@ ## 更新日志 +### 0.2.2 + +- 修复：压缩展开时会保留对象宏中的关键空格，避免 `#define LC (i << 1)` 被误压缩成函数宏。 +- 修复：独立压缩与展开时压缩现在共享同一套 token 输出逻辑，包括用户定义字面量处理。 +- 维护：Rust crate manifest、VS Code 扩展 manifest 和 npm lockfile 的版本均已对齐到 `0.2.2`。 + ### 0.2.0 - 特性：增加了一个配置项 `saveBeforeExpansion`，用于控制是否在展开代码前保存当前文件。默认值为 `true`。