Skip to content

Commit 1972e89

Browse files
KamilbenkiraneKamilbenkirane
authored
support multimodal text messages (#283)
1 parent 24e37dc commit 1972e89

32 files changed

Lines changed: 2351 additions & 424 deletions

File tree

‎.agents/skills/celeste-python/SKILL.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
---
2+
name: celeste-python
3+
description: Use whenever writing, modifying, reviewing, or debugging code involving Celeste, celeste-ai, celeste-python, import celeste, src/celeste, or withceleste app integrations. This includes providers, modalities, models, artifacts, MIME types, parameters, tools, multimodal messages, streaming, structured outputs, protocol/base URL support, OpenSpec changes, and tests. Always use this skill before inventing Celeste types, registries, model catalogs, provider abstractions, request/response shapes, or syntax.
4+
compatibility: Local celeste-python repository skill; no network required.
5+
---
6+
7+
# Celeste Python Coding
8+
9+
Use this skill to stay aligned with the actual Celeste SDK in this repository.
10+
Celeste is evolving, so current source and tests are more reliable than model memory.
11+
12+
## First Step
13+
14+
Read `references/repo-map.md` before making Celeste-related code changes or review claims.
15+
It gives the current public API, internal layering, extension seams, templates, and canonical tests.
16+
17+
Then read only the references relevant to the task:
18+
19+
- App-side integration: `references/public-api.md`
20+
- SDK-internal provider, modality, model, parameter, protocol, streaming, or tool work: `references/sdk-architecture.md`
21+
- Review/debugging or suspicious Celeste code: `references/anti-patterns.md`
22+
- Test selection or final checks: `references/verification.md`
23+
24+
## Source-Of-Truth Order
25+
26+
When sources disagree, follow this order:
27+
28+
1. Current source code in `src/celeste/`
29+
2. Current tests and templates for existing behavior
30+
3. README examples and public exports
31+
4. Active OpenSpec artifacts for intended new behavior and acceptance criteria
32+
5. Notes such as `common_agent_mistakes.md`
33+
6. Model memory
34+
35+
Treat `common_agent_mistakes.md` as advisory. Verify every warning against current code and the task context.
36+
37+
## Route The Task
38+
39+
Classify the task before coding:
40+
41+
- App integration: code outside the SDK consuming Celeste. Prefer public namespaces and public exports. Keep the boundary thin.
42+
- SDK internals: changes inside `src/celeste`, `templates`, or tests. Follow existing modality, provider, protocol, model, mapper, and streaming patterns.
43+
- Review/debugging: identify whether issues are app-side duplication, SDK pattern drift, unsupported model/parameter assumptions, or test gaps.
44+
45+
## App Integration Rules
46+
47+
Use the public API first:
48+
49+
- `celeste.text.*`
50+
- `celeste.images.*`
51+
- `celeste.audio.*`
52+
- `celeste.videos.*`
53+
- `celeste.documents.*`
54+
55+
Use `create_client(...)` when explicit client reuse or explicit `modality`, `operation`, `provider`, `protocol`, `base_url`, or auth configuration is needed.
56+
57+
Do not create app-local duplicates for Celeste-owned concepts unless the user explicitly asks for a temporary compatibility layer. Import and use Celeste types for roles, providers, modalities, operations, artifacts, MIME types, tools, and model discovery.
58+
59+
## SDK Internal Rules
60+
61+
Do not flatten Celeste into a single invented abstraction. The SDK intentionally separates:
62+
63+
- provider auth registration
64+
- modality provider maps
65+
- provider API mixins
66+
- modality-specific provider clients
67+
- per-modality model aggregation
68+
- parameter enums and `ParameterMapper`s
69+
- constraints and optional input type inference
70+
- protocol clients for compatible APIs
71+
72+
Start from nearby existing providers/modalities and templates. Use focused tests as executable documentation.
73+
74+
## Common Failure Modes
75+
76+
Before adding new local code, check whether Celeste already owns the concept.
77+
Watch for:
78+
79+
- raw provider, role, MIME, modality, operation, or input-type strings where Celeste exposes enums
80+
- app-side model allowlists where `list_models(...)` or the host app's existing Celeste discovery flow should be used
81+
- runtime registry patching from app code
82+
- provider SDK request shapes copied directly into app code
83+
- new parameter names without modality enums, `TypedDict` fields, model constraints, and provider mappers
84+
- provider registration changes that update one seam but miss auth, modality maps, model aggregation, or tests
85+
86+
## Verification
87+
88+
Pick focused tests from `references/verification.md` based on the seam touched.
89+
For broader read-only checks, use the repo commands:
90+
91+
- `make lint`
92+
- `make typecheck`
93+
- `make test`
94+
95+
For finalization commands that may rewrite files, use them only when mutation is intended:
96+
97+
- `make format`
98+
- `make ci` (runs lint-fix and format internally)
99+
100+
If you cannot run a relevant command, say so and explain the remaining risk.

‎.agents/skills/celeste-python/evals/evals.json‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
{
2+
"skill_name": "celeste-python",
3+
"evals": [
4+
{
5+
"id": 1,
6+
"prompt": "Integrate Celeste into an app endpoint that generates text, analyzes uploaded images, and lists available text generation models. Keep the integration thin and type-safe.",
7+
"expected_output": "Uses namespace-first public APIs or create_client only where explicit reuse is needed; imports Celeste public types; uses list_models for discovery; avoids local provider/model/role/MIME registries.",
8+
"files": [],
9+
"expectations": [
10+
"Uses celeste.text or celeste.images namespace APIs for app-side operations unless create_client is justified.",
11+
"Uses list_models with Modality and Operation instead of a hard-coded app-side model catalog.",
12+
"Uses Celeste artifacts, MIME enums, Message, Role, Provider, Modality, and Operation where those concepts appear.",
13+
"Does not patch or mutate Celeste internals from app code."
14+
]
15+
},
16+
{
17+
"id": 2,
18+
"prompt": "Add support for a new hypothetical text provider named AcmeAI that exposes an OpenAI-compatible chat completions API. Describe or implement the files and tests needed in celeste-python.",
19+
"expected_output": "Follows SDK seams: Provider enum, provider auth registration, provider API or protocol reuse, modality provider client/map, model definitions and aggregation, parameter mappers where needed, and focused tests.",
20+
"files": [],
21+
"expectations": [
22+
"Identifies Provider enum registration in src/celeste/core.py.",
23+
"Identifies provider auth registration under src/celeste/providers/acmeai/__init__.py.",
24+
"Uses a concrete provider API package or the chat completions protocol seam instead of inventing a flat request stack.",
25+
"Updates modality text provider registration and model aggregation.",
26+
"Defines models with operations and parameter_constraints.",
27+
"Selects focused tests such as test_provider_api_templates.py, test_init.py, test_models.py, and parameter/client tests as relevant."
28+
]
29+
},
30+
{
31+
"id": 3,
32+
"prompt": "Review this Celeste integration: it defines local Provider and Role enums as strings, keeps a hard-coded BACKEND_MODELS list, maps file extensions to raw MIME strings, and registers unknown models at runtime before calling Celeste. Identify what is wrong and how to fix it.",
33+
"expected_output": "Flags app-side duplication and registry mutation, recommends Celeste public enums/types/model discovery/MIME enums, and scopes warnings so SDK-internal registry work is not incorrectly banned.",
34+
"files": [],
35+
"expectations": [
36+
"Flags local Provider and Role enums as duplicates of Celeste-owned concepts.",
37+
"Recommends list_models or existing host discovery instead of a hard-coded model catalog.",
38+
"Recommends Celeste MIME enums and artifact types instead of raw MIME string maps where applicable.",
39+
"Rejects app-side runtime model registry patching while noting SDK-internal model definitions are legitimate.",
40+
"Grounds review claims in current Celeste source or skill references rather than memory alone."
41+
]
42+
},
43+
{
44+
"id": 4,
45+
"prompt": "A chat app wants to send an uploaded image and user text to a Celeste text model with native multimodal input. Show the Celeste message shape it should build.",
46+
"expected_output": "Builds Message(content=[TextPart(...), ImagePart(image=ImageArtifact(...))]) or equivalent ordered MessagePart list; does not pass raw ImageArtifact in content; does not define app-local ImagePart or MessageContent types.",
47+
"files": [],
48+
"expectations": [
49+
"Uses Celeste Message, Role, TextPart, ImagePart, and ImageArtifact.",
50+
"Represents media through ordered MessagePart wrappers inside Message.content.",
51+
"Does not pass raw artifacts or mixed raw lists as Message.content.",
52+
"Does not create app-local duplicate multimodal message models."
53+
]
54+
},
55+
{
56+
"id": 5,
57+
"prompt": "Implement a multi-turn text conversation that sends a previous tool result back to the model. The tool result contains structured JSON.",
58+
"expected_output": "Uses ToolResult as a separate message-list item with structured content in ToolResult.content; does not subclass or wrap it as Message; keeps normal Message.content narrow.",
59+
"files": [],
60+
"expectations": [
61+
"Uses ToolResult directly in the text messages list.",
62+
"Keeps structured tool payloads in ToolResult.content.",
63+
"Does not treat ToolResult as a Message subclass.",
64+
"Does not put structured Pydantic payloads in normal Message.content."
65+
]
66+
},
67+
{
68+
"id": 6,
69+
"prompt": "Configure Celeste to call a custom OpenAI-compatible text endpoint using base_url and protocol. Explain what modalities this supports today.",
70+
"expected_output": "Uses create_client with Modality.TEXT plus protocol/base_url; states protocol-based compatible client support is currently text-only; distinguishes root protocol mixins from text modality protocol clients.",
71+
"files": [],
72+
"expectations": [
73+
"Uses create_client with modality=Modality.TEXT and protocol/base_url.",
74+
"States compatible protocol client support is currently text-only.",
75+
"Does not imply protocol= works for images, audio, videos, documents, or embeddings.",
76+
"Mentions text-specific protocol clients under src/celeste/modalities/text/protocols when discussing SDK internals."
77+
]
78+
},
79+
{
80+
"id": 7,
81+
"prompt": "Review a provider/template change that added extra_headers support only in one provider client file. What else should be checked?",
82+
"expected_output": "Checks templates, provider API mixins, parameter mappers/constraints if public, focused tests, and streaming/telemetry preservation when relevant.",
83+
"files": [],
84+
"expectations": [
85+
"Points to templates as source of truth for generated provider/module shape.",
86+
"Checks provider API mixin and concrete modality provider seams.",
87+
"Considers parameter mapper and model constraint updates if extra_headers is public configuration.",
88+
"Selects focused tests from references/verification.md instead of guessing from memory.",
89+
"Does not run mutating commands unless mutation is intended."
90+
]
91+
}
92+
]
93+
}

‎.agents/skills/celeste-python/references/anti-patterns.md‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Celeste Anti-Patterns
2+
3+
Use this for reviews, debugging, and suspicious Celeste integrations.
4+
5+
## Scope The Rule
6+
7+
Many warnings apply differently depending on context.
8+
9+
App-side integration:
10+
11+
- consume Celeste public APIs
12+
- avoid local duplicates of Celeste concepts
13+
- keep wrappers thin
14+
- prefer `list_models(...)` and existing model discovery
15+
16+
SDK-internal work:
17+
18+
- may update Celeste registries and model definitions
19+
- must follow existing provider/modality/protocol/mapper/test patterns
20+
- should not be blocked by app-side "do not register models" warnings
21+
22+
## Do Not Duplicate Celeste Concepts
23+
24+
Avoid local copies of:
25+
26+
- provider strings when `Provider` applies
27+
- role strings when `Role` applies
28+
- MIME strings when Celeste MIME enums apply
29+
- artifact shapes when `ImageArtifact`, `AudioArtifact`, `VideoArtifact`, or `DocumentArtifact` apply
30+
- message part shapes when `MessageContent`, `MessagePart`, `TextPart`, `ImagePart`, `AudioPart`, `VideoPart`, or `DocumentPart` apply
31+
- model catalogs when `list_models(...)` or existing registry flow applies
32+
- tool result/error wrappers when `ToolOutput` or `ToolError` applies
33+
34+
Local types are fine for truly app-specific transport events or persistence shapes.
35+
36+
## Do Not Bypass Message Semantics
37+
38+
Celeste owns semantic chat message content.
39+
40+
Avoid:
41+
42+
- raw artifacts directly in `Message.content`
43+
- mixed raw lists such as `["text", ImageArtifact(...)]`
44+
- structured Pydantic payloads as normal `Message.content`
45+
- treating `ToolResult` as a `Message` subclass
46+
- hand-rolled provider message serializers in app code or new SDK paths
47+
48+
Use ordered message parts for chat input. Use `ToolResult.content` for structured tool outputs.
49+
50+
## Do Not Invent A Flat Provider Registry
51+
52+
Celeste provider support is intentionally split:
53+
54+
- provider auth registration
55+
- provider enum registration
56+
- provider API mixins
57+
- modality provider maps
58+
- modality-specific provider clients
59+
- per-modality model aggregation
60+
- per-provider model definitions
61+
- parameter mappers and constraints
62+
63+
A new single registry that bypasses these seams is probably wrong.
64+
Directory presence alone is not support; every active provider must be wired through the relevant enum, auth, client, model, map, and test seams.
65+
66+
## Do Not Copy Vendor SDK Shapes Into App Code
67+
68+
Celeste exists to hide provider-specific request/response differences. In app code, avoid OpenAI/Anthropic/Gemini request structures unless the user explicitly asks for provider-specific fallback behavior outside Celeste.
69+
70+
Inside the SDK, provider wire shapes belong in provider API mixins, protocol clients, provider tools, and provider parameter mappers.
71+
72+
## Do Not Add Parameters In Only One Place
73+
74+
New SDK parameters usually need several updates:
75+
76+
- modality parameter enum
77+
- modality `TypedDict`
78+
- model `parameter_constraints`
79+
- provider or protocol mapper
80+
- tests
81+
82+
If only a request-body field was added, the change is probably incomplete.
83+
84+
## Do Not Treat Notes As Canonical
85+
86+
`common_agent_mistakes.md` is useful but not perfect. Verify every warning against:
87+
88+
1. current source
89+
2. current tests/templates
90+
3. public exports and README
91+
92+
Example: app code should not patch Celeste's model registry, but SDK code may legitimately update model definitions or registry behavior.
93+
94+
## Review Checklist
95+
96+
Ask:
97+
98+
- Is this app code or SDK code?
99+
- Is the code using public namespaces where app usage is enough?
100+
- Did it invent local enums or string literals for Celeste-owned concepts?
101+
- Did it duplicate model catalog logic?
102+
- Did it update every provider/modality/model/mapper seam required by the change?
103+
- Did it pick focused tests that match the touched seam?
104+
- Is a warning based on current code, or only on stale memory/notes?

0 commit comments

Comments
 (0)