Skip to content

fix(settings): recreate every service that reads LLM_API_URL - #2518

Open
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:fix/settings-multi-consumer-llm-url
Open

fix(settings): recreate every service that reads LLM_API_URL#2518
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:fix/settings-multi-consumer-llm-url

Conversation

@Hoang130203

Copy link
Copy Markdown

Summary

_match_apply_service() names one owner per .env key. LLM_API_URL is
owned by llama-server, so changing it scheduled llama-server and returned —
while six other services read the same variable straight out of compose:

# docker-compose.base.yml
open-webui:
  OPENAI_API_BASE_URL: "${OPEN_WEBUI_LLM_BASE_URL:-${LLM_API_URL:-http://llama-server:8080}/v1}"
dashboard-api:
  - OLLAMA_URL=${LLM_API_URL:-http://llama-server:8080}
  - LLM_URL=${LLM_URL:-${LLM_API_URL:-http://llama-server:8080}}

plus openclaw, perplexica, privacy-shield and token-spy in their own
compose.yaml.

All of them kept the old endpoint, and the dashboard reported:

services: ['llama-server']
summary : Saved changes are ready to apply to llama-server.

Success, no manual-restart warning, six services still pointed at the previous
URL.

This is the one genuinely silent case in the apply plan. Everywhere else an
unrouted key at least lands in the manual bucket and tells the user to restart
the stack themselves (that is what #2517 is about). Here the user is told the
change is handled, and it is not.

It bites exactly when it matters most: LLM_API_URL is what you change when
you point ODS at a different inference endpoint — an external Ollama, a remote
provider, a relocated llama-server. Chat keeps talking to the old one.

Fix

_EXTRA_APPLY_SERVICES: dict[str, tuple[str, ...]] = {
    "LLM_API_URL": (
        "open-webui", "openclaw", "perplexica", "privacy-shield", "token-spy",
    ),
}

applied after the primary service and filtered through the same
_SETTINGS_APPLY_ALLOWED_SERVICES gate, so this table cannot be used to
schedule something the dashboard is not permitted to recreate. (That gate is
what I got wrong in #2517 and is now asserted here.)

dashboard-api is deliberately not listed: it reads LLM_API_URL, but it
is the process running the apply and is not in the allowed set.

After:

services: llama-server, open-webui, openclaw, perplexica, privacy-shield, token-spy
summary : Saved changes are ready to apply to llama-server, open-webui,
          openclaw, perplexica, privacy-shield, token-spy.

Uninstalled consumers are staged, not scheduled, through the existing
schedule() path — a box without perplexica does not get a phantom service in
its plan.

Test

Five assertions, two of which are guards rather than examples:

  • test_extra_consumers_really_read_the_key parses docker-compose.base.yml
    and each service's compose.yaml, and fails if a service listed in the table
    does not actually reference the key. The table cannot drift from reality.
  • test_extra_consumers_respect_the_allowed_service_gate fails if a listed
    service is not in _SETTINGS_APPLY_ALLOWED_SERVICES.
  • Plus the end-to-end plan, the inactive-services staging path, and a
    single-consumer key (CTX_SIZE) asserted unchanged.

AI Assistance

AI assisted with drafting the table and tests, and wording this description. I
found the case while inventorying apply routing for #2517, confirmed each
consumer by grepping the compose files, and ran the full plan before and after.

Release Lane

  • Stable hotfix targeting release/2.6.x
  • Mainline change targeting main
  • Next-minor work targeting the next feature/minor release
  • Not sure; reviewer should help classify

Stable hotfix reason:

n/a

Changed Surface

  • Docs only
  • Tests only
  • Dashboard UI
  • Dashboard API / host agent
  • Installer / bootstrap / lifecycle
  • Docker Compose / service manifests
  • Model routing / Hermes / capabilities
  • Network exposure / auth / proxy
  • Dependencies / runtime wiring

(One lookup table and four lines at its call site in settings.py, plus tests.)

Risk And Validation

  • Risk level: Low
  • Validation run:
    • git diff --check
    • Markdown/link sanity for docs
    • Focused tests listed below
    • Dashboard lint/test/build
    • Extension audit / compose validation
    • Release-grade fleet or scoped hardware validation
    • Stable-lane patch validation, if targeting release/2.6.x

Commands/results:

$ python3 -m py_compile extensions/services/dashboard-api/settings.py
py OK

$ python3 -m pytest tests/test_settings_env.py -q
81 passed

$ python3 -m pytest tests/test_settings_env.py -q -k MultiConsumer
5 passed, 76 deselected

# behaviour, before -> after:
  before:  services=['llama-server']
  after :  services=['llama-server', 'open-webui', 'openclaw', 'perplexica',
                     'privacy-shield', 'token-spy']
  CTX_SIZE (single consumer) unchanged: ['llama-server']

# the new assertions against origin/main's settings.py:
FAILED TestMultiConsumerApplyRouting::test_llm_api_url_schedules_every_consumer
FAILED TestMultiConsumerApplyRouting::test_extra_consumers_respect_the_allowed_service_gate
FAILED TestMultiConsumerApplyRouting::test_extra_consumers_really_read_the_key
FAILED TestMultiConsumerApplyRouting::test_inactive_extra_consumers_are_staged_not_scheduled
4 failed, 1 passed

Operational Change Check

_compute_env_apply_plan() runs when the dashboard saves settings. Changing
LLM_API_URL now recreates five more containers than before — that is more
disruption per save, and it is the point: those containers were running with a
stale endpoint. Every other key is untouched, asserted by the unchanged
CTX_SIZE case and the 76 other tests in the file. Uninstalled services are
staged rather than started. No write path or schema change.

  • This is not an operational change.
  • This is an operational change and validation is recorded above.
  • This is an operational change and validation is intentionally deferred for:

Notes For Reviewers

I kept this to one key on purpose. The inventory turned up other
multi-consumer variables, but each needs a judgement I would rather you make
than have me guess:

  TIMEZONE          hermes, n8n, open-webui      currently skipped entirely
  UID / GID         hermes, n8n, privacy-shield  changing these mid-life is
                                                 a bigger question than a restart
  EMBEDDING_MODEL   embeddings, open-webui       interacts with the existing
                                                 RAG_* sync logic above it
  ODS_DEVICE_NAME   dashboard-api, ods-proxy, open-webui, tailscale
  BIND_ADDRESS      18 services — already in _MANUAL_RESTART_KEYS, deliberately

LLM_API_URL is the one where the current behaviour is actively misleading
rather than merely conservative, so it is the one worth fixing on its own. Tell
me which of the others you want and I will extend the table — the guard test
means each addition has to be true of the compose files.

Why a table rather than making _match_apply_service return a set. That
was my first instinct, but the function's single-owner return is used to decide
continue versus the manual bucket, and widening it means touching that
control flow and the summary text. A separate additive table leaves the
existing behaviour exactly as it was and is easy to revert. If you would prefer
the refactor, say so and I will send it instead.

Related to #2517, which fixes keys that were unrouted entirely. Independent
files? No — both touch settings.py, so they will need rebasing on each other.
Merge #2517 first and I will rebase this one, or tell me and I will combine
them.

_match_apply_service() names one owner per key. LLM_API_URL is owned by
llama-server, so changing it scheduled llama-server and returned — while
six other services read the same variable from compose:

    open-webui       OPENAI_API_BASE_URL falls back to ${LLM_API_URL}/v1
    openclaw         compose.yaml
    perplexica       compose.yaml
    privacy-shield   compose.yaml
    token-spy        compose.yaml
    dashboard-api    OLLAMA_URL and LLM_URL both fall back to it

All of them kept the old endpoint, and the dashboard reported "Saved
changes are ready to apply to llama-server" — success, with no
manual-restart warning. This is the one silently-wrong case in the apply
plan: everywhere else an unrouted key at least tells the user to restart
the stack themselves.

Adds _EXTRA_APPLY_SERVICES, an explicit table of additional consumers,
applied after the primary service and filtered through the same
_SETTINGS_APPLY_ALLOWED_SERVICES gate. dashboard-api is deliberately not
listed: it is the process running the apply, and it is not in the allowed
set.

Now:

    services: llama-server, open-webui, openclaw, perplexica,
              privacy-shield, token-spy

Uninstalled consumers are staged rather than scheduled, via the existing
schedule() path, so a box without perplexica does not get a phantom
service in its plan.

Five assertions, including one that reads the compose files and fails if
a service listed in the table does not actually reference the key, and
one that fails if a listed service is not in the allowed set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant