From 477afe93096562b55df438f6ebee90a81a383308 Mon Sep 17 00:00:00 2001 From: Andreas Zwinkau Date: Thu, 19 Feb 2026 10:46:47 +0100 Subject: [PATCH 1/3] DR-008-infra first draft --- docs/design_decisions/DR-008-infra.rst | 85 ++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 docs/design_decisions/DR-008-infra.rst diff --git a/docs/design_decisions/DR-008-infra.rst b/docs/design_decisions/DR-008-infra.rst new file mode 100644 index 00000000000..f73f13b1dea --- /dev/null +++ b/docs/design_decisions/DR-008-infra.rst @@ -0,0 +1,85 @@ +.. + Copyright (c) 2026 Contributors to the Eclipse Foundation + + See the NOTICE file(s) distributed with this work for additional + information regarding copyright ownership. + + This program and the accompanying materials are made available under the + terms of the Apache License Version 2.0 which is available at + https://www.apache.org/licenses/LICENSE-2.0 + + SPDX-License-Identifier: Apache-2.0 + +DR-008-Infra: Docs Modularity +================================================================ + +.. dec_rec:: Modularize docs + :id: dec_rec__infra__docs_modularity + :status: proposed + :context: Infrastructure + :decision: tbd + +Context / Problem +----------------- + +There is a proof-of-concept for an alternative documentation build concept introduced in +`tooling PR 95 `_. +It improves the modularity of documentation but drops features of the existing documentation concept. + +Goals and Requirements +^^^^^^^^^^^^^^^^^^^^^^ + +- **Implementation Effort**: Don't spend much one-time effort to implement the change proposed here. +- **Maintenance Effort**: Don't spend much ongoing effort due to increased complexity. +- **Ease of use**: Developers don't like working on documentation, so we should make it easy on them. + +Options Considered +------------------ + +Option S: Single docs() only +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We keep the current docs-as-code concept and drop the PoC proposal. + +Implementation Effort 💚 No additional effort. + +Maintenance Effort 🤔 unclear in relation to PoC. + +Ease of use 💚 No change for developers. + +Option M: Many Bazel targets +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We drop the current approach and switch to the PoC approach. + +Implementation Effort 😡 Requires restructuring all documentation. + +Maintenance Effort 🤔 unclear + +Ease of use 😡 Requires developers to adapt to new workflow. No VSCode LSP integration. + +Option C: Combine the Best of Both +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We integrated the advantages of the PoC into the existing solution maintaining all benefits. + +Implementation Effort 😡 Certainly the most effort + +Maintenance Effort 😡 Certainly the most effort + +Ease of use 💚 No mandatory change for developers at first but extensions? + +Evaluation +---------- + +Here is the summary, how well each option achieves the goals in order of goal importance: + +.. csv-table:: + :header: Goals, Option S, Option M, Option C + :widths: 15, 10, 10, 10 + + Implementation Effort, 💚, 😡, 😡 + Maintenance Effort, 🤔, 🤔, 😡 + Ease of use, 💚, 😡, 💚 + +**Decision:** Option S has no downsides while the alternatives do. From 971375fe83eacfc401faa34c975a18658a6d1596 Mon Sep 17 00:00:00 2001 From: Andreas Zwinkau Date: Mon, 23 Mar 2026 10:56:37 +0100 Subject: [PATCH 2/3] Mention the term "module API" --- docs/design_decisions/DR-008-infra.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design_decisions/DR-008-infra.rst b/docs/design_decisions/DR-008-infra.rst index f73f13b1dea..63ae3a9a60d 100644 --- a/docs/design_decisions/DR-008-infra.rst +++ b/docs/design_decisions/DR-008-infra.rst @@ -47,7 +47,7 @@ Maintenance Effort 🤔 unclear in relation to PoC. Ease of use 💚 No change for developers. -Option M: Many Bazel targets +Option M: Module API - Many Bazel targets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ We drop the current approach and switch to the PoC approach. From 928633614bb3561ea585c10e6df1c810bb8675ec Mon Sep 17 00:00:00 2001 From: Markus Bechter Date: Mon, 23 Mar 2026 18:48:25 +0100 Subject: [PATCH 3/3] Refactoring goals and evaluation --- docs/design_decisions/DR-008-infra.rst | 211 +++++++++++++++++++------ 1 file changed, 165 insertions(+), 46 deletions(-) diff --git a/docs/design_decisions/DR-008-infra.rst b/docs/design_decisions/DR-008-infra.rst index 63ae3a9a60d..f935d48ae30 100644 --- a/docs/design_decisions/DR-008-infra.rst +++ b/docs/design_decisions/DR-008-infra.rst @@ -10,8 +10,8 @@ SPDX-License-Identifier: Apache-2.0 -DR-008-Infra: Docs Modularity -================================================================ +DR-008-Infra: Documentation Modularity +======================================= .. dec_rec:: Modularize docs :id: dec_rec__infra__docs_modularity @@ -19,67 +19,186 @@ DR-008-Infra: Docs Modularity :context: Infrastructure :decision: tbd -Context / Problem ------------------ - -There is a proof-of-concept for an alternative documentation build concept introduced in -`tooling PR 95 `_. -It improves the modularity of documentation but drops features of the existing documentation concept. - -Goals and Requirements -^^^^^^^^^^^^^^^^^^^^^^ - -- **Implementation Effort**: Don't spend much one-time effort to implement the change proposed here. -- **Maintenance Effort**: Don't spend much ongoing effort due to increased complexity. -- **Ease of use**: Developers don't like working on documentation, so we should make it easy on them. +Context and Problem +------------------- + +S-CORE aims to provide modules that can be smoothly integrated into safety-relevant +ECU projects. To support this, S-CORE modules must provide their users with well-defined +artifacts that clearly describe the content of a +`dependable element `_. +The existing documentation build concept does not properly support this. + +A solution for overcoming these limitations is shown in +`tooling PR #95 `_. +It improves the modularity of documentation and allows modules to be built +hermetically by leveraging the features that Bazel provides as a build system. + +Limitations of the Current Approach +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The current documentation build is driven by ``src/incremental.py`` — a single entry +point shared by multiple ``py_binary`` targets. It deliberately escapes the Bazel +sandbox by reading ``BUILD_WORKSPACE_DIRECTORY`` and writing Sphinx output directly +into the workspace's ``_build/`` directory, including deleting a cache file +(``score_source_code_linker_cache.json``) as a side effect. As a result, these targets +only function via ``bazel run``, never ``bazel build``, and parallel invocations can +race on shared workspace state. + +This compromises the following fundamental Bazel features: + +- **Hermeticity** — Actions can read and write arbitrary host filesystem state, so + builds depend on the machine environment rather than only on declared inputs. +- **Reproducibility** — Because outputs land in ``_build/`` (workspace-resident, not + Bazel-tracked), two identical ``bazel run`` invocations starting from different + workspace states can produce different results. +- **Action caching** — Bazel cannot cache or invalidate results it does not know + about; outputs written outside the output tree are invisible to the cache, so + incremental builds may silently serve stale artifacts. +- **Remote execution** — Remote build systems require all inputs and outputs to be + declared; workspace-relative reads and writes cannot be shipped to a remote worker. +- **Remote caching** — Results cannot be shared across machines via a remote cache + because the outputs are never stored in Bazel's Content-Addressable Store (CAS). +- **Parallel safety** — Bazel guarantees that sandboxed actions do not interfere with + each other; deleting ``_build/score_source_code_linker_cache.json`` is an undeclared + side effect that races with concurrent builds or actions. +- **bazel build compatibility** — These targets are restricted to ``bazel run`` only, + since ``BUILD_WORKSPACE_DIRECTORY`` is not set during ``bazel build``, making them + unusable in CI pipelines that rely on standard build commands. +- **Build graph correctness** — Downstream targets cannot depend on outputs that exist + only in the workspace; Bazel's dependency graph is incomplete, so it cannot guarantee + a correct topological execution order. + +How PR #95 Improves the Situation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. **Fully sandboxed Sphinx builds** — + The PR introduces ``sphinx_wrapper.py``, a new Sphinx entry point that accepts + explicit command-line arguments (``--index_file``, ``--output_dir``, ``--config``, + ``--builder``) and is invoked through proper ``ctx.actions.run()`` calls inside + dedicated Bazel rule implementations. All inputs and outputs are declared, so Sphinx + runs entirely within the Bazel sandbox. This restores hermeticity, action caching, + remote execution, remote caching, parallel safety, and full ``bazel build`` + compatibility. + +2. **Proper JSON serialization for external needs** — + Rather than embedding label lists as environment variable strings, the PR writes a + well-formed ``needs_external_needs.json`` file using ``ctx.actions.write()`` with + ``json.encode_indent()``, and passes it as a declared build input. The generated + ``conf.py`` reads this file directly with ``json.load()``. This ensures correct + JSON parsing and reliable cache invalidation when dependencies change. + +3. **Provider-based dependency tracking** — + The PR replaces string-based label manipulation with a set of typed Bazel + providers — ``SphinxSourcesInfo``, ``SphinxModuleInfo``, ``SphinxNeedsInfo``, + ``DependableElementInfo``, and others. Dependency information now flows through + the build graph via properly typed provider fields and transitive ``depset`` + collections, making cross-module references both robust and easy to reason about. + +4. **Composable, purpose-built rules** — + Instead of a single monolithic macro generating many targets at once, the PR offers + a composable set of Bazel rules — ``sphinx_module``, ``dependable_element``, + ``feature_requirements``, ``component_requirements``, ``architectural_design``, + ``safety_analysis``, and more. Each is a proper Bazel rule with well-defined + attributes, making builds more modular, testable, and easier to extend. + +5. **No workspace side effects** — + All build outputs are written to Bazel-managed directories. The PR includes a + dedicated ``sphinx_html_merge.py`` tool that merges HTML from multiple modules + using only declared inputs and outputs, ensuring that no files are created, + modified, or deleted in the source tree during a build. + +6. **Template-driven per-module configuration** — + Rather than relying on a single shared ``conf.py``, the PR generates a tailored + Sphinx configuration for each module from a ``conf.template.py`` template using + ``ctx.actions.expand_template()``. External needs are loaded from the + build-action-produced JSON file, keeping each module's configuration self-contained + and reproducible. + +Goals +----- + +- **Proper support of S-CORE adoption**: Enable teams to structure their documentation + according to the S-CORE process, with dedicated artifact types for requirements, + architectural design, safety analysis, and assumptions of use. +- **Proper support of Bazel**: Ensure documentation builds adhere to Bazel's core + principles — hermeticity, reproducibility, and correct dependency tracking — enabling + action caching, remote caching, remote execution, and parallel builds. +- **Implementation effort**: The one-time cost of migrating existing repositories from + the current ``docs()`` macro to the new rule-based approach, including BUILD file + changes and any necessary restructuring of documentation sources. +- **Maintenance effort**: The ongoing cost of keeping the documentation build + infrastructure working, including dependency management, debugging build failures, + and adapting to changes in Bazel, Sphinx, or the S-CORE process. +- **Ease of use**: How straightforward it is for developers to create, build, preview, + and update documentation as part of their normal development workflow. Options Considered ------------------ -Option S: Single docs() only -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Option S: Single ``docs()`` Only +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -We keep the current docs-as-code concept and drop the PoC proposal. +We keep the current ``docs()`` macro and do not adopt the module-based approach. -Implementation Effort 💚 No additional effort. +- **Proper support of S-CORE adoption** 😡 No dedicated artifact types; teams must + manually structure documentation to match the S-CORE process. +- **Proper support of Bazel** 😡 Sphinx builds escape the sandbox, breaking + hermeticity, action caching, remote execution, and parallel safety. +- **Implementation effort** 💚 No additional effort. +- **Maintenance effort** 💛 Known workarounds remain in place, but no new complexity + is introduced. +- **Ease of use** 💚 No change for developers. -Maintenance Effort 🤔 unclear in relation to PoC. - -Ease of use 💚 No change for developers. - -Option M: Module API - Many Bazel targets +Option M: Module API — Many Bazel Targets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -We drop the current approach and switch to the PoC approach. - -Implementation Effort 😡 Requires restructuring all documentation. +We drop the current approach and switch to the module-based approach. -Maintenance Effort 🤔 unclear - -Ease of use 😡 Requires developers to adapt to new workflow. No VSCode LSP integration. +- **Proper support of S-CORE adoption** 💚 Purpose-built rules for each S-CORE + artifact type (requirements, architectural design, safety analysis, etc.). +- **Proper support of Bazel** 💚 Fully sandboxed builds with declared inputs and + outputs, enabling caching, remote execution, and reproducibility. +- **Implementation effort** 😡 Requires restructuring all existing documentation into + per-module BUILD targets. +- **Maintenance effort** 💚 Documentation builds are standard Bazel actions — no + custom scripts or workspace-escaping workarounds to maintain. +- **Ease of use** 😡 Requires developers to adapt to a new workflow. No VS Code LSP + integration (Esbonio) available. Option C: Combine the Best of Both -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -We integrated the advantages of the PoC into the existing solution maintaining all benefits. - -Implementation Effort 😡 Certainly the most effort - -Maintenance Effort 😡 Certainly the most effort - -Ease of use 💚 No mandatory change for developers at first but extensions? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We integrate the module-based rules into the existing solution, maintaining the current +developer experience while adding S-CORE structure and Bazel correctness where possible. + +- **Proper support of S-CORE adoption** 💛 Possible through gradual adoption, but both + systems must be kept consistent. +- **Proper support of Bazel** 💛 Module-based targets are fully sandboxed, but the + ``docs()`` macro path retains its existing sandbox violations. +- **Implementation effort** 😡 Highest effort — requires implementing and maintaining + both approaches in parallel. +- **Maintenance effort** 😡 Two parallel documentation build paths must be kept working + and consistent. +- **Ease of use** 💚 No mandatory change for developers initially; module-based rules + available as an opt-in extension. Evaluation ---------- -Here is the summary, how well each option achieves the goals in order of goal importance: +Summary of how well each option achieves the goals, in order of importance: .. csv-table:: - :header: Goals, Option S, Option M, Option C - :widths: 15, 10, 10, 10 + :header: Goal, Option S, Option M, Option C + :widths: 25, 10, 10, 10 + + Proper support of S-CORE adoption, 😡, 💚, 💛 + Proper support of Bazel, 😡, 💚, 💛 + Implementation effort, 💚, 😡, 😡 + Maintenance effort, 💛, 💚, 😡 + Ease of use, 💚, 😡, 💚 - Implementation Effort, 💚, 😡, 😡 - Maintenance Effort, 🤔, 🤔, 😡 - Ease of use, 💚, 😡, 💚 +Decision +-------- -**Decision:** Option S has no downsides while the alternatives do. +Option M — adopt the module-based approach and invest in VS Code LSP integration +to restore the developer experience.