Merge develop into master: specs, testing, verification, FAIR/DGX tooling

[montge]

Summary

• OpenSpec workflow: 9 main specs, 22 archived changes, skills-based OPSX commands replacing CLI
• Test coverage: ~150 new test files covering controls, locationd, common, tools, system modules; Hypothesis property tests; algorithm test harness with benchmarks
• Formal verification: CBMC (safety harness), SPIN (msgq protocol), TLA+ (selfdrived state machine), libFuzzer (CAN/safety)
• FAIR integration: DINOv2, SAM 2, CoTracker, DETR, UnSAMFlow wrappers; knowledge distillation framework; QAT pipeline; student models
• DGX Spark: GPU detection, TensorRT benchmarks, DoRA fine-tuning pipeline, model runner
• Shadow device: OnePlus 6 LineageOS setup, camera bridge, remote inference streaming, comparison analysis tools
• Stone Soup: Algorithm comparison framework, voxel grid/Viterbi/octree trackers, track fusion
• Repo hygiene: Pre-commit hooks (ruff, mypy, codespell, openspec-validate), coverage CI workflows, .gitignore cleanup, build artifact removal

Stats

• 227 commits
• 425 files changed
• ~104,800 lines added

Test plan

• Pre-commit hooks pass on all commits
• CI workflows (coverage, MISRA, verification) are correctly configured
• OpenSpec validation passes (openspec validate --strict)
• No secrets or credentials in committed files
• Build artifacts properly gitignored

🤖 Generated with Claude Code

In general, the problem arises because the test encodes an expectation that a domain string only needs to appear as a substring within API_HOST. To avoid incomplete URL substring checks, the test should assert that API_HOST matches the expected value or at least ends with the expected domain, rather than allowing it to appear anywhere.

The best minimally invasive fix here is to change the assertion on line 21 from assert "commadotai.com" in API_HOST to a stricter check such as assert API_HOST.endswith("commadotai.com"). This still allows prefixes like schemes (https://) or subdomains (api.), which are likely present in the real API_HOST constant, but no longer encodes the unsafe “substring anywhere” pattern that CodeQL warns about. We don’t need any new methods, imports, or definitions: it’s just a change to the assertion in TestApiConstants.test_api_host_default in common/tests/test_api.py.

In general, instead of checking that a domain string is a substring of a URL, parse the URL using a standard library and assert properties of the parsed components, especially the hostname. For this test, we should parse API_HOST using urllib.parse.urlparse, then assert on parsed.hostname (and optionally scheme), rather than using 'commadotai.com' in API_HOST.

Concretely, in tools/lib/tests/test_api.py, within class TestAPIHost, change test_default_host so that it uses urlparse(API_HOST).hostname and checks that the hostname equals or ends with the expected domain. We also need to import urlparse (e.g., from urllib.parse import urlparse) at the top of the file. This preserves the intent—verifying that the default API host is on the commadotai.com domain—while avoiding substring checks on the full URL. No other behavior in the tests needs to change.

In general, to fix incomplete URL substring sanitization, you should parse the URL and perform checks on structured components (such as hostname, scheme, and path) instead of checking for substrings within the raw URL string. This prevents attackers from bypassing checks by embedding trusted hostnames in unexpected positions.

For this specific test, we should stop asserting 'accounts.google.com' in url and instead parse url using urllib.parse.urlparse, then assert on parsed.hostname. This keeps the test’s intention—verifying that the redirect points to Google’s OAuth endpoint—without using a substring check that static analysis tools correctly flag as unsafe. Concretely:

• Add an import for urlparse from urllib.parse at the top of tools/lib/tests/test_auth.py.
• In TestAuthRedirectLink.test_google_redirect, replace the substring assertion with parsed = urlparse(url) and assert parsed.hostname == 'accounts.google.com'. You can still keep other substring-based checks for query parameters (client_id=, scope=, etc.), as those are not host sanitization checks.
• Optionally, we can make similar structured checks for GitHub and Apple for consistency, but the CodeQL warning in question is specifically about line 20, so the minimal fix is to adjust that line and add the import.

No changes are needed elsewhere, and functionality of the production code under test remains unchanged; we only refine how the test validates the URL.

In general, to avoid incomplete URL substring checks, tests and code should parse URLs and reason about structured components (scheme, host, path) rather than looking for host substrings anywhere in a string.

Here, we should adjust the test in tools/lib/tests/test_cache.py so it no longer asserts "example.com" in result. Instead, we can assert properties that don’t rely on substring host matching but still validate the intended behavior:

• Keep verifying that URLs (with scheme) are treated as “local” cache entries and that the path part is encoded with underscores.
• Optionally, assert that the cache path is under the provided temporary directory and in a local subdirectory.
• Avoid any host-specific substring checks.

Concretely:

• Edit test_url_with_scheme in tools/lib/tests/test_cache.py.
• Remove the line assert "example.com" in result.
• Optionally strengthen the test with a more structural assertion (e.g., that the basename ends with _path_to_file.txt or that the parent directory is local), but without checking for example.com as a substring.
• No new imports or helper functions are strictly necessary; we can rely on os.path which is already imported.

This change preserves all functional expectations about how cache paths are constructed while eliminating the problematic substring host check.

---

General fix: avoid asserting that a raw host substring may appear anywhere in a “sanitized” URL/path. Instead, assert on a well-defined structure derived from proper URL parsing or on a clearly delimited path component. In tests, this means avoiding "example.com" in result and similar checks, and instead checking for a specific directory or filename pattern that the implementation promises.

Best fix here: update test_http_url so that it no longer requires "cdn.example.org" to appear as a free substring. Instead, assert on the expected structure that cache_path_for_file_path should produce. From the other tests we know:

• For local paths, the cache path starts with cache_dir and contains a local directory, then the path with slashes replaced by underscores (e.g., _data_file.zst).
• For HTTPS/HTTP URLs, the cache path appears to include "local" and the host string and then the path part underscored (as in test_url_with_scheme).

To keep existing behavior while addressing the warning, we can:

• Keep the start and underscore-based filename checks.
• Replace the host substring assertion with a check that the path contains an OS path separator followed by the host as a directory name, e.g. os.path.join(tmpdir, "local", "cdn.example.org") as a prefix. This checks a specific, structured location of the host instead of arbitrary substring use.

Concretely, in tools/lib/tests/test_cache.py, in TestCachePathForFilePath.test_http_url, replace:

      result = cache_path_for_file_path("http://cdn.example.org/data/file.zst", cache_dir=tmpdir)

      assert "cdn.example.org" in result
      assert "_data_file.zst" in result

with a more structured set of assertions, for example:

      url = "http://cdn.example.org/data/file.zst"
      result = cache_path_for_file_path(url, cache_dir=tmpdir)

      expected_prefix = os.path.join(tmpdir, "local", "cdn.example.org")
      assert result.startswith(expected_prefix)
      assert result.endswith("_data_file.zst")

This still validates that the host is encoded into the cache path, but in a deterministic location and via proper path-joining semantics, instead of any arbitrary substring occurrence. It also continues to match the general behavior inferred from the other tests.

No new helper methods are required; we only reuse the existing os import.

In general, fix this by explicitly defining a permissions block that grants only the minimal required access to the GITHUB_TOKEN. For this workflow, the steps only need to read the repository contents (for checkout) and write to the job summary, which does not require any additional API scopes beyond the default. Therefore, setting permissions: contents: read at the workflow or job level is sufficient. No step interacts with pull requests, issues, or performs any repository writes.

The single best fix with minimal functional impact is to add a permissions block at the job level under cbmc-verification in .github/workflows/cbmc.yml. This keeps the restriction tightly scoped to this job without affecting any other jobs that might be added later. Insert the block between timeout-minutes: 15 and steps: as:

permissions:
  contents: read

No imports, methods, or other definitions are needed because this is a YAML configuration change only.

In general, to fix this, add an explicit permissions block either at the workflow root (applies to all jobs) or within the specific job to restrict the GITHUB_TOKEN to the least privileges needed. For this fuzzing workflow, the steps only need to read repository contents (checkout) and upload artifacts (which does not require repository write permissions), so contents: read is sufficient.

The single best fix with no behavior change is to define a job-level permissions block under jobs.fuzz that sets contents: read. This directly addresses the CodeQL warning, documents the minimal permissions, and avoids affecting any other workflows. Concretely, in .github/workflows/fuzz.yml, under jobs: fuzz: and alongside runs-on and timeout-minutes, insert:

permissions:
  contents: read

No additional imports, methods, or other definitions are needed because this is purely a YAML configuration change for the GitHub Actions workflow.

In general, the fix is to explicitly declare a permissions: block limiting the GITHUB_TOKEN to the least privilege necessary. Since this workflow only needs to read repository contents (for actions/checkout) and upload artifacts (which does not require write access to the repo), the minimal appropriate permission is contents: read. Defining this at the workflow (top) level will apply to all jobs that do not override it, covering both spin-verification and spin-simulation.

The single best fix without changing functionality is to add a root-level permissions: block right after the on: section (or after name: and before on:—both are valid, but we’ll place it after on: to minimize line disruption). This will state permissions: contents: read, ensuring the GITHUB_TOKEN cannot write to the repository while still allowing checkout. No additional imports or methods are needed; this is a pure YAML configuration change within .github/workflows/spin.yml. Only that file and that region need modification.

To fix the problem, explicitly define restricted GITHUB_TOKEN permissions for this workflow or for each job, instead of relying on repository defaults. The least-privilege baseline recommended by GitHub and CodeQL for read-only workflows is permissions: contents: read at minimum. Since this workflow also uploads artifacts but does not interact with pull requests or issues, contents: read at the workflow root is sufficient.

The single best fix without changing functionality is to add a workflow-level permissions block right after the name: field. This will apply to all jobs (spin-verification and spin-simulation) which currently have no permissions block of their own. No other code or steps need to change. Concretely, in .github/workflows/spin.yml, add:

permissions:
  contents: read

right after line 15 (name: SPIN Protocol Verification). No imports or additional methods are required, as this is a YAML configuration change only.

To fix the problem, explicitly restrict the GITHUB_TOKEN permissions for this workflow to the minimum necessary. Since the job only checks out code and downloads/runs tools, it only needs read access to repository contents, and possibly no other scopes. The best fix is to add a permissions: block that sets contents: read. This can be done either at the workflow root (applying to all jobs) or inside the tlaplus job; placing it at the root is simpler and documents the default for any future jobs added to this workflow.

Concretely, edit .github/workflows/tlaplus.yml and insert a top-level permissions: block after the name: TLA+ Verification line (line 12). The block should be:

permissions:
  contents: read

No additional imports, methods, or definitions are needed, as this is purely a YAML configuration change in the workflow file and does not alter existing functionality of the steps.

[github-actions]

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

[github-actions]

This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.