Skip to content

Make hardware profile an explicit runtime policy - #2

Merged
GuideboardLabs merged 1 commit into
GuideboardLabs:mainfrom
Ozhiaki:codex/hardware-runtime-profile
May 14, 2026
Merged

Make hardware profile an explicit runtime policy#2
GuideboardLabs merged 1 commit into
GuideboardLabs:mainfrom
Ozhiaki:codex/hardware-runtime-profile

Conversation

@TheEditor

Copy link
Copy Markdown
Contributor

Summary

Adds a first-class active hardware/runtime profile layer. Oathweaver can now resolve a named hardware profile at startup, initialize scheduler budgets from that profile, and pass normalized capacity policy into inference-router diagnostics.

This makes the project’s local runtime assumptions explicit without changing model routing intent or adding machine-specific profiles.

Why

Oathweaver already has hardware-aware pieces: scheduler budgets, benchmark profiles, model-routing diagnostics, premium model locks, and conservative fit estimation. Those pieces were not yet connected through one durable active runtime policy.

This PR makes the target-machine policy explicit so future users can describe local capacity in config instead of relying on implicit workstation assumptions.

Changes

  • Adds structured hardware profile config in SourceCode/configs/hardware_profiles.json.
  • Adds active profile resolution via OATHWEAVER_HARDWARE_PROFILE.
  • Converts resolved profiles into HardwareBudgetProfile for scheduler/resource-budget use.
  • Converts resolved profiles into router policy for estimate_fit() and validate_config().
  • Includes active hardware profile details in pipeline and replay diagnostics.
  • Allows kernel commands to apply structured hardware profiles by name.
  • Keeps enforcement advisory-first: profile mismatches warn rather than hard-block startup or model calls.

Example Profile Shape

{
  "name": "local_cuda_workstation",
  "display_name": "Local CUDA Workstation",
  "description": "Example profile for a local CUDA machine. Values are illustrative and should be adjusted by the operator.",
  "hardware": {
    "system_ram_gb": 32,
    "gpu_backend": "cuda",
    "gpu_vram_gb": 12,
    "unified_memory": false
  },
  "scheduler": {
    "max_context_tokens": 8192,
    "warning_context_tokens": 6144,
    "max_stage_context_tokens": 2200,
    "max_parallel_models": 1,
    "max_active_model_calls": 1,
    "on_deck_depth": 1,
    "warm_depth": 0,
    "allow_neural_prefetch": false
  },
  "inference": {
    "preferred_backends": ["llama.cpp", "ollama"],
    "default_keep_alive": "5m",
    "heavy_keep_alive": "0",
    "release_heavy_after_call": true,
    "max_loaded_models": 1
  },
  "model_policy": {
    "normal_max_b": 9,
    "heavy_max_b": 14,
    "premium_min_b": 24,
    "allow_premium": false,
    "premium_requires_manual": true,
    "allow_14b_with_warning": true,
    "reject_heavier_fallbacks": false
  },
  "lane_caps": {
    "chat_layer": {
      "max_context_tokens": 6144,
      "max_parallel_agents": 1
    }
  },
  "validation": {
    "startup_mode": "warn",
    "strict_mode_available": true,
    "warn_on_missing_models": true,
    "warn_on_unreachable_backends": true,
    "warn_on_context_over_cap": true,
    "warn_on_parallelism_over_cap": true,
    "warn_on_premium_auto_escalation": true
  }
}

Validation

  • python3 -m unittest tests.test_hardware_profiles tests.test_inference_router_compat tests.test_phase12_cag_bench_integration
  • python3 -m unittest discover tests
  • python3 -m json.tool SourceCode/configs/hardware_profiles.json >/dev/null
  • git diff --check

@GuideboardLabs
GuideboardLabs marked this pull request as ready for review May 14, 2026 14:42
@GuideboardLabs
GuideboardLabs merged commit 72bc06f into GuideboardLabs:main May 14, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants