Tech debt: unify local and cloud LLM configuration around profiles

[enyst]

Summary

Today the UI has to special-case local vs cloud LLM configuration:

• local /settings is profile-oriented (LLM Profiles)
• cloud /settings is still raw LLM Settings
• footer/menu wording also has to branch by backend mode

This is functionally correct, but it creates product and implementation friction. The same conceptual surface (choosing/configuring how the agent uses an LLM) presents itself differently depending on backend kind.

Problem

We currently expose two mental models:

1. Local: saved LLM profiles that can be created, edited, and activated.
2. Cloud: direct raw LLM settings.

That leaks backend details into the product UX and forces frontend copy / navigation / settings surfaces to diverge.

Goal

Find a way to present one user-facing profile model across local and cloud, even if the underlying storage/mechanics differ.

Possible direction

One possible approach:

• keep profiles as the UI abstraction everywhere
• on cloud, store profiles client-side (or in a lightweight app-layer store)
• when starting a conversation or saving cloud settings, materialize/flatten the selected profile into the raw llm payload expected by the cloud path

This would let us:

• avoid backend-specific copy like LLM Profiles vs LLM Settings
• keep footer/settings navigation semantics consistent
• let users think in terms of named configurations regardless of backend kind

Open questions

• Where should cloud-side profile data live (browser localStorage, app backend, agent-server proxy, etc.)?
• Should cloud support exactly one effective profile at a time, or multiple named saved profiles like local?
• How should profile secrets/API keys be stored and redacted safely on cloud?
• Should profile switching affect only new conversations, or also live conversations when supported?
• Can we converge the local and cloud settings APIs enough that the UI no longer branches on backend kind?

Why now

Recent UI work exposed the awkwardness here directly:

• the local settings page needed to be renamed to LLM Profiles
• local footer links needed profile wording
• cloud still has to keep LLM Settings

That is a reasonable short-term fix, but it is a sign that the conceptual model should probably be unified.

This issue was created by an AI agent (OpenHands) on behalf of the user.

[enyst]

@OpenHands look at the open PRs and issues on OpenHands/OpenHands and agent-sdk repos. Is there a PR or issue on profiles on Cloud? what is the status? Respond directly on this issue.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[enyst]

I checked the current open work in both OpenHands/OpenHands and OpenHands/software-agent-sdk.

OpenHands/OpenHands (Cloud / SaaS side)

There is active Cloud/SaaS profile work open there:

• Issue #14405 — Enable LLM Profiles for SaaS (Org-Level Storage)
  • Status: open
  • This is the clearest tracker for "profiles on Cloud" right now.
• PR #14403 — feat(settings): add saved LLM profiles - SaaS
  • Status: open draft
  • CI/checks were green on the latest run, but it is still draft / blocked pending review.
• PR #14406 — feat: Add org-level LLM profiles for SaaS users
  • Status: open draft
  • Also green on the latest CI run, but still draft / not merged.
• Issue #11977 — [PRD] LLM Profile Management
  • Status: open
  • This looks like the broader product/design umbrella for the profiles model.

OpenHands/software-agent-sdk (agent-server / SDK side)

I did not find an open Cloud-specific profiles tracker there. What I found instead is that the foundational profile API work has already landed:

• PR #3129 — feat(profiles): LLM profile management API — merged
• PR #3161 — feat(profiles): Add cipher encryption for LLM profile secrets — merged
• PR #3173 — feat(profiles): add active_profile tracking to profiles API — merged
• PR #3188 — Add LLM profile switch tool — merged

My read on status

So the status seems to be:

• SDK / agent-server support for profiles is already in place.
• Cloud/SaaS profile support is being worked on in OpenHands/OpenHands, not in agent-sdk.
• The Cloud work is not merged yet; it is currently represented by open draft PRs plus an open SaaS/org-level tracking issue.

That makes this issue feel current/relevant: agent-canvas can probably align with the emerging model, but the Cloud-side canonical implementation still appears to be in flight upstream.

This comment was created by an AI agent (OpenHands) on behalf of the user.

[openhands-ai]

Summary of work since my last update:

• Answered the question by investigating related work in:
  • OpenHands/OpenHands
  • OpenHands/software-agent-sdk
• Posted a direct response on agent-canvas issue Tech debt: unify local and cloud LLM configuration around profiles #533 with the findings.

Findings restated:

• There is active Cloud/SaaS profile work in OpenHands/OpenHands:
  • Issue #14405 — “Enable LLM Profiles for SaaS (Org-Level Storage)” — open
  • PR #14403 — “feat(settings): add saved LLM profiles - SaaS” — open draft
  • PR #14406 — “feat: Add org-level LLM profiles for SaaS users” — open draft
  • Issue #11977 — broader LLM profile management PRD — open
• In OpenHands/software-agent-sdk, I did not find an open Cloud-specific profiles tracker.
• The SDK/profile foundations are already merged there:
  • PR #3129 — LLM profile management API
  • PR #3161 — encrypted profile secrets
  • PR #3173 — active_profile tracking
  • PR #3188 — profile switch tool

Completion check:

• Investigated open issues/PRs in both requested repos
• Determined whether there is Cloud-profile-related work and its status
• Responded directly on the issue
• Included AI disclosure in the posted comment

Conciseness check:

• No code changes were made
• No extraneous repository changes were introduced

Bottom line:

• The status is that Cloud/SaaS profile support is actively in progress upstream in OpenHands/OpenHands, but not merged yet, while the agent-sdk side already has the necessary profile infrastructure merged.

[VascoSch92]

Hey @enyst

I think there is a little bit of confusion after all the changes.

Moreover, it seems that profiles are not working as expected.

For example, I cannot use in main the /model command to see my profiles and if I cannot use the switch button to switch between profiles. (see the video).

I'm working on that ;-)

problem_model_command.mov

[enyst]

Thank you @VascoSch92 , yes, I’m a tad confused, hah. 😅 I’d love your opinion about the open PR on using profile names, I don’t want it to make your work harder! So I didn’t merge it, happy to close if you want. Tagged you there.

My biggest worry overall, though, in this issue, is that we don’t introduce if openhands this, if saas that again! This would be bad architecture, error prone - and not only the buzzword “error prone”, but for real, there’s been an endless array of issues both for Open Source users and on the Cloud when this stuff takes a life of its own -, and off-putting for contributors.

And I fear I just did this if thing! But I plan to fix it; the way I see it is:

• we are adding profiles support on the cloud
• fix the page so that it’s the same here, client-side
• remove the ugly stuff.

[VascoSch92]

Hey @enyst

A brief recap:

• on both cloud and here we want to have 3 entry point to switch LLM profile: /model NAME, button, ask the LLM
• with the PR: feat/fix(profiles): per-conversation /switch_llm in chat #575 I'm adding the 3 here in canvas.
• in cloud this should be already the case (in OSS) and I have just to test it for SaaS
• Sorry I discover that a lot of PRs are actually overlapping (for example my old one: feat(chat): add switch-profile button next to input #419 and yours fix: prefer llm profile name in chat footer #532 ). So it was easier to created a new one from scratch: feat/fix(profiles): per-conversation /switch_llm in chat #575. I hope it is not a problem.

[enyst]

Awesome, this sounds great, thank you! I’ll wait with this issue until we can make sure it’s harmonized across backends.