Skip to content

Commit 09030e8

Browse files
olivermeyerclaude
andcommitted
feat(database): context-aware defaults for all public fns
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 9470248 commit 09030e8

4 files changed

Lines changed: 294 additions & 44 deletions

File tree

README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,51 @@ All public library functions (`logging_initialize`, `sentry_initialize`, `boot`,
8181
`load_modules`, etc.) accept an optional `context` keyword argument and fall
8282
back to `get_context()` when it is `None`.
8383

84+
### Database
85+
86+
Once a context is configured via `set_context()`, all database functions work
87+
with no arguments — the URL and pool settings are read from the context:
88+
89+
```python
90+
from aignostics_foundry_core.database import init_engine, cli_run_with_db, with_engine
91+
92+
# Zero-arg engine init — reads MYPROJECT_DB_URL, _DB_POOL_SIZE, etc. from env
93+
init_engine()
94+
95+
# CLI helper — initialises engine, runs coroutine, disposes engine
96+
cli_run_with_db(my_async_func)
97+
98+
99+
# Background job decorator — engine initialised before each invocation
100+
@with_engine
101+
async def my_job(): ...
102+
103+
104+
# Override for a secondary database
105+
@with_engine(db_url="postgresql+asyncpg://user:pass@host/secondary")
106+
async def my_other_job(): ...
107+
```
108+
109+
`FoundryContext.from_package()` activates database configuration automatically
110+
when the following environment variables are present:
111+
112+
| Variable | Required | Description |
113+
|---|---|---|
114+
| `{PREFIX}DB_URL` | yes (to activate) | Full database connection URL |
115+
| `{PREFIX}DB_POOL_SIZE` | no | Connection pool size (default `10`) |
116+
| `{PREFIX}DB_MAX_OVERFLOW` | no | Max pool overflow (default `10`) |
117+
| `{PREFIX}DB_POOL_TIMEOUT` | no | Pool wait timeout in seconds (default `30.0`) |
118+
| `{PREFIX}DB_NAME` | no | Override database name in the URL path |
119+
120+
In tests, construct `DatabaseSettings` directly instead of setting env vars:
121+
122+
```python
123+
from aignostics_foundry_core.database import DatabaseSettings
124+
from tests.conftest import make_context
125+
126+
ctx = make_context(database=DatabaseSettings(_env_prefix="TEST_DB_", url="sqlite+aiosqlite:///test.db"))
127+
```
128+
84129
### Health API
85130

86131
```python

src/aignostics_foundry_core/AGENTS.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ This file provides an overview of all modules in `aignostics_foundry_core`, thei
1717
| **log** | Configurable loguru logging initialisation | `logging_initialize(filter_func=None, *, context=None)`, `LogSettings` (env-prefix configurable), `InterceptHandler` for stdlib-to-loguru bridging |
1818
| **sentry** | Configurable Sentry integration | `sentry_initialize(integrations, *, context=None)`, `SentrySettings` (env-prefix configurable), `set_sentry_user(user, role_claim)` for Auth0 user context |
1919
| **service** | FastAPI-injectable base service | `BaseService` ABC with `get_service()` (cached per-class FastAPI `Depends` factory), `key()`, and abstract `health()` / `info()` methods; concrete subclasses implement health checks and module info |
20-
| **database** | Async SQLAlchemy session management + DB settings | `DatabaseSettings` (`OpaqueSettings` subclass; env prefix defaults to `{ctx.env_prefix}DB_`; `get_url()` with optional `db_name` substitution); `init_engine(db_url, pool_size, max_overflow, pool_timeout)`, `dispose_engine()`, `get_db_session()` (FastAPI dependency), `execute_with_session(func, …)`, `cli_run_with_db(func, …, db_url)`, `cli_run_with_engine(func, …, db_url)`, `with_engine(db_url)` decorator factory; auto-resets engine after `fork()` |
20+
| **database** | Async SQLAlchemy session management + DB settings | `DatabaseSettings` (`OpaqueSettings` subclass; env prefix defaults to `{ctx.env_prefix}DB_`; `get_url()` with optional `db_name` substitution); `init_engine(db_url=None, pool_size=None, max_overflow=None, pool_timeout=None)` — all params optional, fall back to active context when `None`; `dispose_engine()`, `get_db_session()` (FastAPI dependency), `execute_with_session(func, …)`, `cli_run_with_db(func, …, db_url=None)`, `cli_run_with_engine(func, …, db_url=None)`, `with_engine` dual-mode decorator (supports `@with_engine`, `@with_engine()`, `@with_engine(db_url=…)`); auto-resets engine after `fork()` |
2121
| **cli** | Typer CLI preparation utilities | `prepare_cli(cli, epilog, *, context=None)` — discovers and registers subcommands via `locate_implementations`, sets epilog recursively, installs `no_args_is_help` workaround; `no_args_is_help_workaround(ctx)` — raises `typer.Exit` when no subcommand is invoked |
2222
| **boot** | Application / library boot sequence | `boot(context, sentry_integrations, log_filter, show_cmdline)` — runs once per process: parses `--env` CLI args, initialises logging and Sentry, amends the SSL trust chain via *truststore* and *certifi*, and logs boot/shutdown messages |
2323
| **user_agent** | Parameterised HTTP user-agent string builder | `user_agent(project_name, version, repository_url)` — builds `{project_name}-python-sdk/{version} (…)` string including platform info, current test, and GitHub Actions run URL |
@@ -321,15 +321,15 @@ This file provides an overview of all modules in `aignostics_foundry_core`, thei
321321

322322
**Async SQLAlchemy session management**
323323

324-
- **Purpose**: Manages a process-level async database engine singleton, providing session injection for FastAPI routes, background jobs, and CLI commands. All Bridge-specific settings are replaced with explicit parameters.
324+
- **Purpose**: Manages a process-level async database engine singleton, providing session injection for FastAPI routes, background jobs, and CLI commands. All public functions accept optional DB-config params and fall back to the active `FoundryContext.database` when they are `None`.
325325
- **Key Features**:
326-
- `init_engine(db_url, pool_size=10, max_overflow=10, pool_timeout=30)` — initialises the global `AsyncEngine` and `async_sessionmaker`; subsequent calls are silent no-ops. Pool parameters are omitted automatically for SQLite (which does not use `QueuePool`).
326+
- `init_engine(db_url=None, pool_size=None, max_overflow=None, pool_timeout=None)` — initialises the global `AsyncEngine` and `async_sessionmaker`; subsequent calls are silent no-ops. When `db_url` is `None`, the URL and pool settings are resolved from `get_context().database`; raises `RuntimeError` if no context is installed or `ctx.database` is `None`. Pool parameters are omitted automatically for SQLite (which does not use `QueuePool`).
327327
- `dispose_engine()` — async; disposes the engine; called during application shutdown.
328328
- `get_db_session()` — async generator; yields an `AsyncSession`; raises `RuntimeError` if engine not initialised. Use as a FastAPI `Depends` target.
329329
- `execute_with_session(async_func, *args, **kwargs)` — async; runs `async_func` with a session injected as the `session` keyword argument. For background jobs and CLI helpers.
330-
- `cli_run_with_db(async_func, *args, db_url, pool_size, max_overflow, pool_timeout, **kwargs)` — synchronous wrapper: initialises engine, runs the coroutine, then disposes. For CLI commands.
331-
- `cli_run_with_engine(async_func, *args, db_url, pool_size, max_overflow, pool_timeout, **kwargs)` — like `cli_run_with_db` but does not inject a session; for jobs that manage sessions themselves.
332-
- `with_engine(db_url, pool_size, max_overflow, pool_timeout)` — decorator factory; wraps an async function to initialise the engine before execution. For long-lived workers; does **not** dispose after running.
330+
- `cli_run_with_db(async_func, *args, db_url=None, pool_size=None, max_overflow=None, pool_timeout=None, **kwargs)` — synchronous wrapper: initialises engine, runs the coroutine, then disposes. All DB-config params optional; fall back to context when `None`. For CLI commands.
331+
- `cli_run_with_engine(async_func, *args, db_url=None, pool_size=None, max_overflow=None, pool_timeout=None, **kwargs)` — like `cli_run_with_db` but does not inject a session; for jobs that manage sessions themselves.
332+
- `with_engine` — dual-mode decorator; supports `@with_engine` (no-parens), `@with_engine()` (empty parens), and `@with_engine(db_url=…, …)` (explicit params). All params optional; fall back to context when absent. For long-lived workers; does **not** dispose after running.
333333
- Fork safety: `multiprocessing.util.register_after_fork` resets the engine in child processes automatically.
334334
- **Location**: `aignostics_foundry_core/database.py`
335335
- **Dependencies**: `sqlalchemy[asyncio]>=2,<3`, `asyncpg>=0.29,<1` (mandatory); `loguru` for structured logging

src/aignostics_foundry_core/database.py

Lines changed: 120 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
import functools
1515
import multiprocessing.util
1616
import urllib.parse
17-
from collections.abc import AsyncGenerator, Callable
17+
from collections.abc import AsyncGenerator
1818
from typing import Any
1919

2020
from loguru import logger
@@ -120,11 +120,56 @@ class _DatabaseModuleSentinel:
120120
multiprocessing.util.register_after_fork(_module_sentinel, lambda _obj: _reset_engine_after_fork())
121121

122122

123+
_DEFAULT_POOL_SIZE = 10
124+
_DEFAULT_MAX_OVERFLOW = 10
125+
_DEFAULT_POOL_TIMEOUT = 30.0
126+
127+
128+
def _resolve_db_params(
129+
db_url: str | None,
130+
pool_size: int | None,
131+
max_overflow: int | None,
132+
pool_timeout: float | None,
133+
) -> tuple[str, int, int, float]:
134+
"""Resolve database connection parameters, falling back to the active context.
135+
136+
When ``db_url`` is ``None``, all four values are sourced from
137+
``get_context().database``. When ``db_url`` is provided, any ``None`` pool
138+
params are replaced by their module-level defaults.
139+
140+
Returns:
141+
A tuple of ``(db_url, pool_size, max_overflow, pool_timeout)``.
142+
143+
Raises:
144+
RuntimeError: If ``db_url`` is ``None`` and no context is installed, or
145+
the context has no ``database`` configured.
146+
"""
147+
if db_url is None:
148+
from aignostics_foundry_core.foundry import get_context # noqa: PLC0415
149+
150+
ctx = get_context()
151+
if ctx.database is None:
152+
msg = f"No database URL configured. Set {ctx.env_prefix}DB_URL or pass db_url explicitly."
153+
raise RuntimeError(msg)
154+
return (
155+
ctx.database.get_url(),
156+
pool_size if pool_size is not None else ctx.database.pool_size,
157+
max_overflow if max_overflow is not None else ctx.database.max_overflow,
158+
pool_timeout if pool_timeout is not None else ctx.database.pool_timeout,
159+
)
160+
return (
161+
db_url,
162+
pool_size if pool_size is not None else _DEFAULT_POOL_SIZE,
163+
max_overflow if max_overflow is not None else _DEFAULT_MAX_OVERFLOW,
164+
pool_timeout if pool_timeout is not None else _DEFAULT_POOL_TIMEOUT,
165+
)
166+
167+
123168
def init_engine(
124-
db_url: str,
125-
pool_size: int = 10,
126-
max_overflow: int = 10,
127-
pool_timeout: float = 30,
169+
db_url: str | None = None,
170+
pool_size: int | None = None,
171+
max_overflow: int | None = None,
172+
pool_timeout: float | None = None,
128173
) -> None:
129174
"""Initialize the database engine singleton.
130175
@@ -135,21 +180,33 @@ def init_engine(
135180
For multiprocessing: Engine is automatically reset in child processes via
136181
multiprocessing.util.register_after_fork().
137182
183+
When ``db_url`` is ``None``, the URL and pool settings are resolved from the
184+
active :class:`~aignostics_foundry_core.foundry.FoundryContext`. A
185+
:exc:`RuntimeError` is raised if no context is installed or the context has no
186+
``database`` configured.
187+
138188
Args:
139189
db_url: Database connection URL (e.g. ``postgresql+asyncpg://user:pass@host/db``).
190+
When ``None``, resolved from the active context's ``database`` settings.
140191
pool_size: Number of connections to keep in the pool. Ignored for dialects that
141-
do not support QueuePool (e.g. SQLite).
192+
do not support QueuePool (e.g. SQLite). Defaults to the context value or 10.
142193
max_overflow: Number of additional connections above pool_size. Ignored for
143-
dialects that do not support QueuePool.
194+
dialects that do not support QueuePool. Defaults to the context value or 10.
144195
pool_timeout: Seconds to wait for a connection from the pool. Ignored for
145-
dialects that do not support QueuePool.
196+
dialects that do not support QueuePool. Defaults to the context value or 30.
197+
198+
Raises:
199+
RuntimeError: If ``db_url`` is ``None`` and no context is installed, or the
200+
context has no ``database`` configured.
146201
"""
147202
global _engine, _async_session_maker # noqa: PLW0603
148203

149204
if _engine is not None:
150205
logger.trace("Database engine already initialized, reusing existing engine and connection pool.")
151206
return # Already initialized
152207

208+
db_url, pool_size, max_overflow, pool_timeout = _resolve_db_params(db_url, pool_size, max_overflow, pool_timeout)
209+
153210
logger.trace(
154211
"Initializing global database engine with pool_size={}, max_overflow={}, pool_timeout={}",
155212
pool_size,
@@ -248,10 +305,10 @@ async def execute_with_session(async_func: Any, *args: Any, **kwargs: Any) -> An
248305
def cli_run_with_db(
249306
async_func: Any, # noqa: ANN401
250307
*args: Any, # noqa: ANN401
251-
db_url: str,
252-
pool_size: int = 10,
253-
max_overflow: int = 10,
254-
pool_timeout: float = 30,
308+
db_url: str | None = None,
309+
pool_size: int | None = None,
310+
max_overflow: int | None = None,
311+
pool_timeout: float | None = None,
255312
**kwargs: Any, # noqa: ANN401
256313
) -> Any: # noqa: ANN401
257314
"""Run an async database function from a synchronous CLI context.
@@ -261,10 +318,14 @@ def cli_run_with_db(
261318
262319
NOT for use in long-lived processes (API, workers) - use @with_engine decorator instead.
263320
321+
When ``db_url`` is ``None``, the URL and pool settings are resolved from the active
322+
:class:`~aignostics_foundry_core.foundry.FoundryContext` (same behaviour as
323+
:func:`init_engine`).
324+
264325
Args:
265326
async_func: The async function to run (receives ``session`` as a keyword argument).
266327
*args: Positional arguments forwarded to ``async_func``.
267-
db_url: Database connection URL.
328+
db_url: Database connection URL. When ``None``, resolved from the active context.
268329
pool_size: Connection pool size (ignored for SQLite).
269330
max_overflow: Max overflow connections (ignored for SQLite).
270331
pool_timeout: Pool wait timeout in seconds (ignored for SQLite).
@@ -291,10 +352,10 @@ def cli_run_with_db(
291352
def cli_run_with_engine(
292353
async_func: Any, # noqa: ANN401
293354
*args: Any, # noqa: ANN401
294-
db_url: str,
295-
pool_size: int = 10,
296-
max_overflow: int = 10,
297-
pool_timeout: float = 30,
355+
db_url: str | None = None,
356+
pool_size: int | None = None,
357+
max_overflow: int | None = None,
358+
pool_timeout: float | None = None,
298359
**kwargs: Any, # noqa: ANN401
299360
) -> Any: # noqa: ANN401
300361
"""Run an async function with initialized database engine from a synchronous CLI context.
@@ -304,10 +365,14 @@ def cli_run_with_engine(
304365
305366
NOT for use in long-lived processes (API, workers) - use @with_engine decorator instead.
306367
368+
When ``db_url`` is ``None``, the URL and pool settings are resolved from the active
369+
:class:`~aignostics_foundry_core.foundry.FoundryContext` (same behaviour as
370+
:func:`init_engine`).
371+
307372
Args:
308373
async_func: The async function to run (does not require a session parameter).
309374
*args: Positional arguments forwarded to ``async_func``.
310-
db_url: Database connection URL.
375+
db_url: Database connection URL. When ``None``, resolved from the active context.
311376
pool_size: Connection pool size (ignored for SQLite).
312377
max_overflow: Max overflow connections (ignored for SQLite).
313378
pool_timeout: Pool wait timeout in seconds (ignored for SQLite).
@@ -331,49 +396,66 @@ def cli_run_with_engine(
331396

332397

333398
def with_engine(
334-
db_url: str,
335-
pool_size: int = 10,
336-
max_overflow: int = 10,
337-
pool_timeout: float = 30,
338-
) -> Callable[[Any], Any]:
339-
"""Decorator factory to ensure database engine is initialized for async functions.
399+
func: Any | None = None, # noqa: ANN401
400+
*,
401+
db_url: str | None = None,
402+
pool_size: int | None = None,
403+
max_overflow: int | None = None,
404+
pool_timeout: float | None = None,
405+
) -> Any: # noqa: ANN401
406+
"""Decorator (or decorator factory) to ensure database engine is initialized for async functions.
407+
408+
Supports two calling conventions:
340409
341-
This decorator wraps an async function to automatically initialize the database
342-
engine singleton before execution. The connection pool persists across all jobs
343-
in the process for efficiency. Useful for background jobs and workers.
410+
* ``@with_engine`` — no-parens form; resolves URL and pool settings from the
411+
active :class:`~aignostics_foundry_core.foundry.FoundryContext`.
412+
* ``@with_engine()`` or ``@with_engine(db_url=..., ...)`` — explicit-parens form;
413+
any omitted params are resolved from the active context.
414+
415+
The connection pool persists across all jobs in the process for efficiency.
416+
Useful for background jobs and workers.
344417
345418
For multiprocessing: Engine is automatically reset in child processes via
346419
multiprocessing.util.register_after_fork().
347420
348421
Args:
349-
db_url: Database connection URL.
422+
func: The async function to decorate (only when used as ``@with_engine``
423+
without parentheses). Do not pass explicitly.
424+
db_url: Database connection URL. When ``None``, resolved from the active context.
350425
pool_size: Connection pool size (ignored for SQLite).
351426
max_overflow: Max overflow connections (ignored for SQLite).
352427
pool_timeout: Pool wait timeout in seconds (ignored for SQLite).
353428
354429
Returns:
355-
A decorator that wraps an async function with engine initialization.
430+
The decorated async function (no-parens form) or a decorator (parens form).
356431
357-
Example:
358-
@with_engine(db_url="postgresql+asyncpg://user:pass@host/db")
432+
Example::
433+
434+
# Context-aware — no arguments needed once set_context() is called:
435+
@with_engine
359436
async def my_job():
360437
result = await execute_with_session(some_db_operation)
361438
return result
439+
440+
441+
# Explicit URL (e.g. secondary database):
442+
@with_engine(db_url="postgresql+asyncpg://user:pass@host/db")
443+
async def my_other_job(): ...
362444
"""
363445

364-
def decorator(func: Any) -> Any: # noqa: ANN401
365-
func_name = getattr(func, "__name__", str(func))
446+
def decorator(f: Any) -> Any: # noqa: ANN401
447+
func_name = getattr(f, "__name__", str(f))
366448
logger.trace("Applying with_engine decorator to function {}", func_name)
367449

368-
@functools.wraps(func)
450+
@functools.wraps(f)
369451
async def wrapper(*args: Any, **kwargs: Any) -> Any: # noqa: ANN401
370452
logger.trace("Initializing database engine in with_engine wrapper for function {}", func_name)
371453
init_engine(db_url=db_url, pool_size=pool_size, max_overflow=max_overflow, pool_timeout=pool_timeout)
372454
logger.debug("Database engine initialized in with_engine wrapper for function {}", func_name)
373455

374456
try:
375457
logger.trace("Executing function {} within with_engine wrapper", func_name)
376-
result = await func(*args, **kwargs)
458+
result = await f(*args, **kwargs)
377459
logger.trace("Successfully executed function {} within with_engine wrapper", func_name)
378460
return result
379461
except Exception:
@@ -382,4 +464,6 @@ async def wrapper(*args: Any, **kwargs: Any) -> Any: # noqa: ANN401
382464

383465
return wrapper
384466

385-
return decorator
467+
if func is not None: # called as @with_engine (no parens)
468+
return decorator(func)
469+
return decorator # called as @with_engine() or @with_engine(db_url=...)

0 commit comments

Comments
 (0)