feat: add LSP server for editor integration by jimisola · Pull Request #323 · reqstool/reqstool-client

jimisola · 2026-03-15T20:25:06Z

Important

This branch is based on feat/313-sqlite-storage (#321) but targets main. PR #321 must be merged first before this PR can be merged.

Summary

Add a Python LSP server to reqstool-client using pygls, backed by the SQLite pipeline from #313. The server provides editor-agnostic support for reqstool annotations in Java, Python, TypeScript, and JavaScript.

Depends on: #321 (SQLite storage)

Implemented features

Hover: Show requirement/SVC details when hovering over @Requirements/@SVCs annotations + YAML field descriptions from JSON schemas
Diagnostics: Red squiggles on unknown/deprecated/obsolete requirement/SVC IDs + YAML schema validation
Completion: Autocomplete requirement/SVC IDs inside annotations + enum values in YAML files (significance, categories, lifecycle, verification, etc.)
Go-to-definition: Navigate from annotations in source code to YAML definitions and vice versa
Document symbols: Outline view for requirements.yml, software_verification_cases.yml, manual_verification_results.yml with cross-references
Multi-project workspaces: Discover and track multiple reqstool projects (e.g., Gradle multi-module with system + microservices)
File watching: Auto-reload on static file changes (requirements.yml, software_verification_cases.yml, manual_verification_results.yml, reqstool_config.yml) + manual refresh command for remote dependency changes
Language support: Java/Python (source-level @Requirements/@SVCs annotations) + TypeScript/JavaScript (JSDoc @Requirements/@SVCs tags)

Architecture

Editor (VS Code / Neovim / IntelliJ / etc.)
  ↕ LSP protocol (stdio)
ReqstoolLanguageServer (pygls)
  → WorkspaceManager (manages multiple projects)
      → ProjectState[] (one per reqstool project found)
          → build_database() pipeline (reused from #313)
          → RequirementsRepository (reused from #313)
  → Features: hover, diagnostics, completion, go-to-definition, document symbols

New CLI command

pip install reqstool[lsp]   # install optional LSP dependencies
reqstool lsp                # start LSP server in stdio mode

New files

File	Purpose
`src/reqstool/lsp/annotation_parser.py`	Regex-based annotation detection (Java/Python decorators + JSDoc tags)
`src/reqstool/lsp/project_state.py`	Wraps `build_database()` pipeline for a single reqstool project
`src/reqstool/lsp/root_discovery.py`	Finds root reqstool projects in workspace folders
`src/reqstool/lsp/workspace_manager.py`	Per-folder isolation managing ProjectState instances
`src/reqstool/lsp/server.py`	pygls LanguageServer with lifecycle, file watching, refresh
`src/reqstool/lsp/yaml_schema.py`	JSON Schema loading, field descriptions, enum values
`src/reqstool/lsp/features/hover.py`	Hover for source annotations + YAML fields
`src/reqstool/lsp/features/diagnostics.py`	Source + YAML schema diagnostics
`src/reqstool/lsp/features/completion.py`	ID + YAML enum completion
`src/reqstool/lsp/features/definition.py`	Source ↔ YAML go-to-definition
`src/reqstool/lsp/features/document_symbols.py`	YAML outline view

Test coverage

133 new LSP unit tests, 433 total unit tests — all passing.

Closes

Closes feat: Language Server Protocol (LSP) for editor integration #314

Test plan

jimisola · 2026-03-15T20:36:09Z

@Jonas-Werne Iterating over the plan now: https://github.com/reqstool/reqstool-client/blob/feat/314-lsp-server/docs/lsp-server-plan.md

…nd multi-root workspace support (#314) Adds refined Q5 root project discovery algorithm, document symbols feature (Step 9), separate root_discovery.py module, workspace/didChangeWorkspaceFolders support, and YAML schema utility (Step 10). Signed-off-by: jimisola <jimisola@users.noreply.github.com>

Move pygls/lsprotocol to project.optional-dependencies instead of main dependencies. Users install with `pip install reqstool[lsp]` only when they need the LSP server; CI pipelines keep the lighter base install. Signed-off-by: jimisola <jimisola@users.noreply.github.com>

…annotation parser (#314) - Add pygls/lsprotocol as optional [lsp] extra in pyproject.toml - Add `reqstool lsp` subcommand with ImportError guard for missing extra - Create annotation_parser module detecting @Requirements/@svcs in Java/Python (source decorators) and TypeScript/JavaScript (JSDoc tags) - Add 33 unit tests for annotation parser covering both syntaxes, position lookup, completion context, and multi-line annotations Signed-off-by: jimisola <jimisola@users.noreply.github.com>

- ProjectState wraps build_database() pipeline for a single reqstool project with query helpers for requirements, SVCs, and MVRs - Root discovery finds root projects in workspace folders by parsing requirements.yml metadata and building a local reference graph - WorkspaceManager provides per-folder isolation with add/remove/rebuild operations and file-to-project resolution - Add 27 unit tests covering project lifecycle, root discovery algorithm, and workspace management Signed-off-by: jimisola <jimisola@users.noreply.github.com>

… command (#314) - ReqstoolLanguageServer (pygls subclass) with workspace manager integration - Lifecycle handlers: initialized, shutdown, didOpen/Change/Save/Close - Workspace folder change tracking (add/remove folders dynamically) - File watcher for static YAML files triggers project rebuild - reqstool.refresh command for manual rebuild of all projects - Diagnostic publishing placeholders for Step 6 Signed-off-by: jimisola <jimisola@users.noreply.github.com>

- Hover on @Requirements/@svcs annotations shows requirement/SVC details - Hover on YAML fields shows JSON Schema descriptions - YAML schema utilities: load schemas, resolve $ref, get field descriptions/enum values - Wire hover handler into LSP server Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- Source code diagnostics: unknown requirement/SVC IDs, deprecated/obsolete lifecycle warnings - YAML diagnostics: parse errors and JSON Schema validation against reqstool schemas - Wire diagnostics into server on didOpen, didChange, didSave, and rebuild events Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- Source completion: offer requirement/SVC IDs inside @Requirements/@svcs annotations - YAML completion: offer enum values for fields like significance, variant, categories - Wire completion handler into server with trigger characters Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- Source → YAML: navigate from @Requirements/@svcs annotations to YAML definitions - YAML → YAML: navigate from requirement IDs to SVC references and vice versa - Wire definition handler into LSP server Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- Outline view for requirements.yml, software_verification_cases.yml, manual_verification_results.yml - Shows requirement/SVC/MVR items with titles, significance, and cross-references as children - Wire document symbol handler into LSP server Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

_find_id_in_yaml only matched `id: <raw_id>` lines, so cross-reference navigation (REQ→SVC, SVC→MVR) returned empty results because IDs appear inside array fields like `requirement_ids: ["REQ_PASS"]`. Add _find_reference_in_yaml with a \b word-boundary pattern and strengthen the integration test to assert actual navigation results. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

Add location_type and location_uri columns to urn_metadata table to record where each URN's data originated (local, git, maven, pypi). Carry resolved file paths through CombinedRawDataset so the LSP go-to-definition handler uses actual paths from reqstool_config.yml instead of hard-coded filenames. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

Add regression-python fixture project with requirements, SVCs, MVRs, annotations, and test results for end-to-end LSP integration tests. Configure pytest-asyncio and add test client infrastructure. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

…#314) - Accept --stdio flag (passed by VS Code LSP client) in lsp subcommand - Add --tcp, --host, --port args for TCP transport support - Wrap start_server() call with exception handler in command.py - Wrap server.start_io()/start_tcp() with exception logging in server.py

…_path (#314)

…okens, codeAction LSP features (#314) - Add reqstool/details custom request for structured REQ/SVC/MVR data - Extend hover with "Open Details" command link - Add textDocument/codeLens (verification status above annotations) - Add textDocument/inlayHint (title inline after ID) - Add textDocument/references (find all usages across open docs + YAML) - Add workspace/symbol (quick-search REQ/SVC IDs) - Add textDocument/semanticTokens/full (color-code deprecated/obsolete) - Add textDocument/codeAction (quick fixes for unknown/deprecated IDs) - Add get_mvr() and get_yaml_paths() to ProjectState - Add _get() and _first_project() shared helpers to server.py Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

… references (#314) - get_requirement_details: adds `references` (cross-refs) and `implementations` (annotation impls) - get_svc_details: adds `test_annotations` and `test_results` (automated test status per annotation) - RequirementsRepository: adds get_test_results_for_svc for targeted per-SVC test result lookup - ProjectState: exposes get_impl_annotations_for_req, get_test_annotations_for_svc, get_test_results_for_svc Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

…s LSP features (#314) - completion: filter DEPRECATED/OBSOLETE IDs from suggestions - codeLens: add ⚠/✕ lifecycle badge per ID; pass full ids list in command args - hover: show implementation count for requirements; tests passed/failed/missing and MVR counts for SVCs - details: add test_summary aggregate, enrich requirement_ids with title+lifecycle_state, add source_paths to all responses - semanticTokens: 4 distinct token types in lifecycle order — reqstoolDraft(0), reqstoolValid(1), reqstoolDeprecated(2), reqstoolObsolete(3) Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- Add static lsp.adoc covering all LSP capabilities, commands, and annotation languages - Add reqstool-lsp.openrpc.json (OpenRPC 1.2.6) as formal spec for the custom reqstool/details method - Fix details response: return id+urn separately (urn = project URN only, not composite UrnId) - Update nav.adoc to include LSP Server page - Update CLAUDE.md with LSP documentation guidance Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

jimisola · 2026-03-19T09:23:19Z

Note: LSP variant changes needed

PR #328 removes variant as a behavioral gate and makes it optional. The LSP module (src/reqstool/lsp/) is not on main yet, but when it lands it will need these updates:

lsp/root_discovery.py: Change DiscoveredProject.variant: VARIANTS → Optional[VARIANTS]. On ValueError, set variant = None instead of returning None.
lsp/workspace_manager.py: Handle root.variant being None in logging (root.variant.value if root.variant else None).

Existing if project.variant == VARIANTS.EXTERNAL: continue logic remains correct since None != VARIANTS.EXTERNAL.

…314) Extract LSP try/except handling into Command.command_lsp() to bring main() from complexity 13 down to the allowed maximum of 10. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

…collection (#314) - Fix test_get_requirement_details_fields: result["urn"] is the URN portion only ("ms-001"), not the full urn:id; assert result["id"] instead - Add norecursedirs = tests/fixtures to prevent pytest collecting fixture source files that import project-specific modules Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

…test expectations (#314) pygls 2.x _with_semantic_tokens() uses the registered options object directly as the `legend` field when constructing ServerCapabilities. Passing SemanticTokensOptions caused a nested SemanticTokensOptions as the legend, breaking serialization at LSP initialize time. Fix: pass SemanticTokensLegend to @server.feature() directly so pygls can wrap it correctly into SemanticTokensOptions. Also fix integration test expectations: completion correctly excludes deprecated/obsolete REQs and SVCs (REQ_SKIPPED_TEST, REQ_OBSOLETE, SVC_050, SVC_070) — update tests to assert this behaviour. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

- CLAUDE.md: keep both LSP Documentation section (this branch) and Design Decisions section (main/feat/324) - database.py: keep location_type/location_uri columns from this branch and adopt variant optionality (variant.value if variant else None) from main/feat/324 Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

jimisola mentioned this pull request Mar 15, 2026

feat: Language Server Protocol (LSP) for editor integration #314

Open

jimisola requested a review from Jonas-Werne March 15, 2026 20:26

jimisola self-assigned this Mar 15, 2026

jimisola added 20 commits March 18, 2026 23:21

docs: add LSP server implementation plan (#314)

94a8dcb

style: apply black formatting and fix flake8 unused import (#314)

7ae81a5

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

fix: replace PyYAML with ruamel.yaml in LSP diagnostics (#314)

a8b6f99

feat: add --log-file option to lsp command for debugging (#314)

9b58fc5

fix: match source files to project via workspace folder, not reqstool…

5331f5d

…_path (#314)

jimisola force-pushed the feat/314-lsp-server branch from 21e5991 to 7fc2a9c Compare March 18, 2026 22:22

jimisola added 3 commits March 18, 2026 23:27

fix: reduce cyclomatic complexity of main() below flake8 C901 limit (#…

d08eb0b

…314) Extract LSP try/except handling into Command.command_lsp() to bring main() from complexity 13 down to the allowed maximum of 10. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

jimisola added 3 commits March 19, 2026 22:50

jimisola marked this pull request as ready for review March 19, 2026 22:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add LSP server for editor integration#323

feat: add LSP server for editor integration#323
jimisola wants to merge 27 commits intomainfrom
feat/314-lsp-server

jimisola commented Mar 15, 2026 •

edited

Loading

Uh oh!

jimisola commented Mar 15, 2026

Uh oh!

jimisola commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jimisola commented Mar 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implemented features

Architecture

New CLI command

New files

Test coverage

Closes

Related

Test plan

Uh oh!

jimisola commented Mar 15, 2026

Uh oh!

jimisola commented Mar 19, 2026

Note: LSP variant changes needed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jimisola commented Mar 15, 2026 •

edited

Loading