Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,10 @@ gateway.

- **Activity**: the per-request log of what the gateway served, with filters.
Use it to inspect individual requests, their models, and their outcomes.
Requests the gateway refused are logged too, so filtering to the `error`
status shows what is being dropped, including requests rejected for having no
pricing under `require_pricing`. Diagnostic detail stays in the gateway logs
rather than the browser.
- **Usage**: aggregate usage and analytics, showing spend and volume over time,
broken down by model and by user.

Expand Down
21 changes: 20 additions & 1 deletion src/gateway/api/routes/_passthrough.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,9 +219,28 @@ async def run_passthrough(
and pricing_required_but_missing(pricing, require_pricing=config.require_pricing)
):
await refund_reservation(db, reservation)
no_pricing_detail = no_pricing_error_detail(model)
# Record the rejection so dropped traffic is visible in the activity
# log and countable as an error, rather than only reaching the
# operator as a user complaint. cost stays null: nothing was spent.
await log_writer.put(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the third hand-built UsageLog literal in this file (also the success row at ~263 and the provider-error row at ~290), all eleven fields repeated. A local _usage_row(status, **overrides) helper would make the next field addition a one-line change instead of three sites to keep in sync.

UsageLog(
id=str(uuid.uuid4()),
api_key_id=api_key_id,
user_id=user_id,
timestamp=datetime.now(UTC),
model=resolved.model,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logs model=resolved.model (bare, e.g. text-embedding-3-small) where _pipeline.py logs model=model, the full selector (openai:gpt-4o). Each matches its own file's success path, so this PR does not introduce the split, but it does extend it to the new error rows: in the Activity log a rejected embeddings row and a rejected chat row render their Model column differently, and the model filter matches one form or the other.

Not necessarily worth changing here, but the new tests do not pin it either (the chat test asserts the model string, the passthrough one does not), so asserting the expected embeddings model would at least record which form is intended.

provider=resolved.instance,
endpoint=endpoint,
status="error",
error_message=no_pricing_detail,
latency_ms=_elapsed_ms(started_at),
counts_toward_budget=not budget_exempt,
)
)
raise HTTPException(
status_code=status.HTTP_402_PAYMENT_REQUIRED,
detail=no_pricing_error_detail(model),
detail=no_pricing_detail,
)

# Model access control (per-key). The reservation is already taken above (the
Expand Down
21 changes: 20 additions & 1 deletion src/gateway/api/routes/_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -710,9 +710,28 @@ async def resolve_request_context(
# cost=null when unpriced.
if not budget_exempt and pricing_required_but_missing(gate_pricing, require_pricing=config.require_pricing):
await refund_reservation(db, reservation)
no_pricing_detail = no_pricing_error_detail(model)
# Record the rejection before raising. Gateway-side rejections used to
# leave no trace, so an operator who flipped require_pricing on had no
# way to see that live traffic was being dropped (only a user complaint
# would tell them). The row carries cost=null: nothing was spent, so it
# never affects spend or the budget, it just makes the drop visible in
# the activity log and countable as an error.
await log_usage(
db=db,
log_writer=log_writer,
api_key_id=api_key_id,
model=model,
provider=gate_instance,
endpoint=adapter.endpoint,
user_id=user_id,
error=no_pricing_detail,
latency_ms=_elapsed_ms(started_at),
counts_toward_budget=not budget_exempt,
)
raise adapter.error(
402,
no_pricing_error_detail(model),
no_pricing_detail,
ErrorKind.INVALID_REQUEST,
)

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading