Skip to content

Rename Behaviors to Requirements - #2465

Open
harry-rhesis wants to merge 1 commit into
mainfrom
feat/rename-requirements
Open

Rename Behaviors to Requirements#2465
harry-rhesis wants to merge 1 commit into
mainfrom
feat/rename-requirements

Conversation

@harry-rhesis

Copy link
Copy Markdown
Contributor

Purpose

Rename the Behavior domain entity to Requirement across the platform. This branch carries both the implementation plan and the work itself.

Replaces #2199, which held the original proposal and the review discussion that settled the strategy. Plan: docs/proposals/rename-behaviors-to-requirements.md.

The first commit is the plan alone. Implementation lands on top.

What Changed

  • Added the implementation plan: locked decisions, full inventory, stored-data migration list, execution order, and verification gates

Decisions locked

# Decision Rationale
1 Single atomic cutover, not a phased rollout Behavior is chained through org bootstrap seed data, the Garak taxonomy (backend and SDK copies), the MCP tool contract, and stats/insights. A deprecation window means holding two vocabularies in sync across that chain for months, and the failure mode is a wrong label rather than a crash, so drift sits unnoticed.
2 DB rename folded in (former "Option B") Deferring it leaves the ORM mapping Requirement onto table behavior indefinitely, and every raw-SQL site keeps the old name. Two migrations for one end state.
3 No aliases, no dual-read, no dual API paths /behaviors has no external announcement or SLA. Building a compatibility layer in one release and tearing it out in another is cost with no consumer.
4 Frontend /behaviors keeps a permanent redirect The one exception to #3. It is a UI route with user bookmarks, not an API contract, and it costs one line.
5 Entity values are not renamed Reliability, Robustness, Compliance are referenced by name from initial_data.json, garak/taxonomy.py, and garak/detectors.yaml. The entity is renamed; its instances are not.
6 Stored historical values get a backfill, not dual-read Every one lands in the same Alembic revision as the schema rename.

Measured footprint

Verified against main at d5d4b11d0.

Measure Count
Files containing behavio (excluding node_modules, build output) 701
Matching lines ~6,540
Of those, in frozen migrations and changelogs ~200

Sharp edges the plan covers

Database objects a table rename does not follow

Postgres does not rewrite function bodies, policy names, index names, or view output aliases.

  • alembic/delete_user_and_organization_data.sql is a PL/pgSQL function with a hardcoded table-name array and DELETE FROM behavior. After a rename it references a table that no longer exists, and it fails only at org-deletion time. Needs CREATE OR REPLACE FUNCTION.
  • Stats views alias b.name AS behavior_name. That alias is baked into the view definition, so the views need DROP and CREATE, not a rename.
  • app/services/annotations.py runs raw SQL joining behavior with behavior_id / behavior_name aliases. No ORM rename touches it.
  • Also: RLS policy names, FK and index names embedding behavior, the auto-RLS trigger table list, and the reference copy in alembic/row_level_security.sql.

Stored data that breaks silently

  • architect_session.plan_data persists the Architect plan with keys behaviors, behavior_metric_mappings, behavior. Renaming the Pydantic fields in sdk/.../architect/plan.py without backfilling breaks deserialization of every saved session. This is the sharpest remaining path.
  • Preflight check id behavior_metric_coverage, MetricsSource.BEHAVIOR, stats mode behavior and metric key behavior_pass_rates, type_lookup row Behavior, PlanCategory.BEHAVIOR.
  • RBAC is narrower than first framed: built-in roles compute permission sets from code, so they follow the enum rename automatically. Custom roles have role_permission rows keyed on permission.id, so the migration must UPDATE permission SET name in place. Delete-and-insert orphans every custom grant.

Surfaces absent from the original inventory

  • skills/rhesis/ (21 files, ~200 refs). Published and installed by users via npx skills add rhesis-ai/rhesis. references/tool-catalog.md mirrors the MCP tool names and must move with mcp_tools.yaml.
  • penelope/ (19 files), a separate package with prompt templates and 9 examples.
  • Vocabulary collision. skills/rhesis/references/requirements-workflow.md and docs/content/docs/agent-skill/requirements.mdx already use "Requirements" to mean the input PRD, with behaviors as the derived entity. After the rename that reads as "build requirements from your requirements." Plan recommends renaming the input concept to "spec".
  • docs/sphinx/source/rhesis.entities.rst has .. autoclass:: rhesis.sdk.entities.Behavior, which breaks the SDK API docs build if missed.
  • docs/content/glossary/glossary-terms.jsonl (35 lines of structured data, separate from the glossary MDX pages), and docs/src/public/diagrams/user-journey.excalidraw, which needs re-export rather than a text edit.
  • docs/content/contribute/backend/odata-guide.mdx (39 refs). OData navigation property names come from ORM relationship names, so $filter=behavior/name is a user-facing contract that changes.
  • SDK _BEHAVIOR_TOOLS in agents/architect/agent.py hardcodes MCP tool names, so it must land in the same commit as mcp_tools.yaml or Architect silently stops recognizing entity tool calls.
  • tests/k6/common.js load-tests /behaviors/, which would 404 after the cutover and read as a perf regression.
  • apps/frontend/src/config/test-templates.generated.ts must be regenerated via scripts/generate-templates.js, not hand-edited.
  • NEXT_PUBLIC_BEHAVIORS_EMPTY_STATE_VIDEO_URL and ..._ARTICLE_URLS are set in the deploy environment, outside any diff.

Execution shape

701 files cannot be a 400-line PR. The plan proposes:

  1. 4 prep PRs, each small and behavior-preserving: resolve the vocabulary collision, extract hardcoded string literals into the existing frontend and backend constants modules, and add a regression test for the plan_data round-trip before touching it.
  2. One atomic cutover with a single Alembic revision and ordered commits (migration, backend, MCP + SDK tools together, SDK, frontend, tests, tooling, contract docs). Not splittable without a broken intermediate state.
  3. Follow-ups for docs prose, glossary, notebooks, and prompt-text triage.

Plus a release checklist for the deploy-environment env vars, the migrate-before-deploy ordering, and the SDK major bump.

Verification gates

Line-by-line review cannot carry a 6,000-line diff, so the plan leans on mechanical checks: a source sweep against an explicit allowlist of things we deliberately do not rename, eight Postgres catalog queries (including pg_proc.prosrc, which is what catches the delete-org function), row-level assertions against a restored production snapshot, behavioral checks for the silent-failure paths, and a tested alembic downgrade rehearsal.

Additional Context

  • Replaces Add implementation plan for Behaviors to Requirements rename #2199. The strategy discussion and the original phased proposal live in that thread.
  • Breaking for API and SDK consumers by design, per decision npm warn deprecated inflight@1.0.6 #3.
  • The schema rename is not backward-compatible with old app code, so this needs a short maintenance window rather than a rolling deploy.
  • Two corrections came out of re-verifying against current main: app/crud.py is now the package app/crud/, and the explorer JSONB marker is already migrated to test_set.explorer_row (7dd69fe35db5), which removes what was the worst silent-failure risk.

Still open

  1. Vocabulary collision: "spec" for the PRD input, or accept the overload?
  2. test-templates.yml prompt English: change and regenerate, or leave?
  3. Metric prompt English ("refusal behavior", "forbidden behaviors"): separate evaluated change, or leave indefinitely?
  4. Confirm the maintenance window is acceptable.

Testing

The plan's verification gates apply as implementation lands. Nothing to test in the plan commit itself.

@peqy

peqy Bot commented Aug 12, 2026

Copy link
Copy Markdown

Looks good — this plan is detailed and pragmatic (esp. the silent-failure inventory + idempotent backfill + downgrade rehearsal).

One concrete addition I’d suggest: include an explicit OpenAPI gate in “Verification gates” (e.g. assert generated /openapi.json has no /behaviors paths / behavio strings). In this repo the MCP tooling looks up paths/methods from the FastAPI OpenAPI schema, so an OpenAPI/MCP mismatch can fail tool wiring even if the code compiles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant