Skip to content

fix: Autolog Mechanics #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
monoxgas merged 4 commits into from
Jul 12, 2025
Merged

fix: Autolog Mechanics #99

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
acee9cb
Remove parameter logging from tasks. Add useful serialization check f…
monoxgas Jul 7, 2025
de5d9e1
Merge remote-tracking branch 'origin/main' into fix/autolog-updates
monoxgas Jul 7, 2025
f3d0d1d
Apply seems_useful_to_serialize to task outputs
monoxgas Jul 7, 2025
f9a33cf
Update docs to align with code changes
monoxgas Jul 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/intro.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ async def execute_command(command: str) -> str:
stdout, stderr = await process.communicate()

# Log additional information
dn.log_param("exit_code", process.returncode)
dn.log_output("exit_code", process.returncode)

result = stdout.decode() if process.returncode == 0 else stderr.decode()
return result # Automatically logged as output
Expand Down
87 changes: 24 additions & 63 deletions docs/sdk/main.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1280,12 +1280,7 @@ def log_outputs(
### log\_param

```python
log_param(
key: str,
value: JsonValue,
*,
to: ToObject = "task-or-run",
) -> None
log_param(key: str, value: JsonValue) -> None
```

Log a single parameter to the current task or run.
Expand All @@ -1309,13 +1304,6 @@ with dreadnode.run("my_run"):
* **`value`**
(`JsonValue`)
–The value of the parameter.
* **`to`**
(`ToObject`, default:
`'task-or-run'`
)
–The target object to log the parameter to. Can be "task-or-run" or "run".
Defaults to "task-or-run". If "task-or-run", the parameter will be logged
to the current task or run, whichever is the nearest ancestor.

<Accordion title="Source code in dreadnode/main.py" icon="code">
```python
Expand All @@ -1324,8 +1312,6 @@ def log_param(
self,
key: str,
value: JsonValue,
*,
to: ToObject = "task-or-run",
) -> None:
"""
Log a single parameter to the current task or run.
Expand All @@ -1343,11 +1329,8 @@ def log_param(
Args:
key: The name of the parameter.
value: The value of the parameter.
to: The target object to log the parameter to. Can be "task-or-run" or "run".
Defaults to "task-or-run". If "task-or-run", the parameter will be logged
to the current task or run, whichever is the nearest ancestor.
"""
self.log_params(to=to, **{key: value})
self.log_params(**{key: value})
```


Expand All @@ -1356,9 +1339,7 @@ def log_param(
### log\_params

```python
log_params(
to: ToObject = "run", **params: JsonValue
) -> None
log_params(**params: JsonValue) -> None
```

Log multiple parameters to the current task or run.
Expand All @@ -1379,13 +1360,6 @@ with dreadnode.run("my_run"):

**Parameters:**

* **`to`**
(`ToObject`, default:
`'run'`
)
–The target object to log the parameters to. Can be "task-or-run" or "run".
Defaults to "task-or-run". If "task-or-run", the parameters will be logged
to the current task or run, whichever is the nearest ancestor.
* **`**params`**
(`JsonValue`, default:
`{}`
Expand All @@ -1395,7 +1369,7 @@ with dreadnode.run("my_run"):
<Accordion title="Source code in dreadnode/main.py" icon="code">
```python
@handle_internal_errors()
def log_params(self, to: ToObject = "run", **params: JsonValue) -> None:
def log_params(self, **params: JsonValue) -> None:
"""
Log multiple parameters to the current task or run.

Expand All @@ -1413,19 +1387,11 @@ def log_params(self, to: ToObject = "run", **params: JsonValue) -> None:
~~~

Args:
to: The target object to log the parameters to. Can be "task-or-run" or "run".
Defaults to "task-or-run". If "task-or-run", the parameters will be logged
to the current task or run, whichever is the nearest ancestor.
**params: The parameters to log. Each parameter is a key-value pair.
"""
task = current_task_span.get()
run = current_run_span.get()

target = (task or run) if to == "task-or-run" else run
if target is None:
raise RuntimeError("log_params() must be called within a run")

target.log_params(**params)
if (run := current_run_span.get()) is None:
raise RuntimeError("Parameters must be logged within a run")
run.log_params(**params)
```


Expand Down Expand Up @@ -1543,7 +1509,7 @@ with dreadnode.run("my_run"):
(`bool`, default:
`True`
)
–Whether to automatically log task inputs, outputs, and execution metrics if unspecified.
–Whether to automatically log task inputs, outputs, and execution metrics if otherwise unspecified.
* **`**attributes`**
(`Any`, default:
`{}`
Expand Down Expand Up @@ -1592,7 +1558,7 @@ def run(
project: The project name to associate the run with. If not provided,
the project passed to `configure()` will be used, or the
run will be associated with a default project.
autolog: Whether to automatically log task inputs, outputs, and execution metrics if unspecified.
autolog: Whether to automatically log task inputs, outputs, and execution metrics if otherwise unspecified.
**attributes: Additional attributes to attach to the run span.

Returns:
Expand Down Expand Up @@ -1919,11 +1885,11 @@ task(
scorers: None = None,
name: str | None = None,
label: str | None = None,
log_params: Sequence[str] | bool = False,
log_inputs: Sequence[str]
| bool
| Inherited = INHERITED,
log_output: bool | Inherited = INHERITED,
log_execution_metrics: bool = False,
tags: Sequence[str] | None = None,
**attributes: Any,
) -> TaskDecorator
Expand All @@ -1935,11 +1901,11 @@ task(
scorers: Sequence[Scorer[R] | ScorerCallable[R]],
name: str | None = None,
label: str | None = None,
log_params: Sequence[str] | bool = False,
log_inputs: Sequence[str]
| bool
| Inherited = INHERITED,
log_output: bool | Inherited = INHERITED,
log_execution_metrics: bool = False,
tags: Sequence[str] | None = None,
**attributes: Any,
) -> ScoredTaskDecorator[R]
Expand All @@ -1952,11 +1918,11 @@ task(
| None = None,
name: str | None = None,
label: str | None = None,
log_params: Sequence[str] | bool = False,
log_inputs: Sequence[str]
| bool
| Inherited = INHERITED,
log_output: bool | Inherited = INHERITED,
log_execution_metrics: bool = False,
tags: Sequence[str] | None = None,
**attributes: Any,
) -> TaskDecorator
Expand Down Expand Up @@ -1992,21 +1958,21 @@ await my_task(2)
`None`
)
–The label of the task - useful for filtering in the UI.
* **`log_params`**
(`Sequence[str] | bool`, default:
`False`
)
–Whether to log all, or specific, incoming arguments to the function as parameters.
* **`log_inputs`**
(`Sequence[str] | bool | Inherited`, default:
`INHERITED`
)
–Whether to log all, or specific, incoming arguments to the function as inputs.
–Log all, or specific, incoming arguments to the function as inputs.
* **`log_output`**
(`bool | Inherited`, default:
`INHERITED`
)
–Whether to log the result of the function as an output.
–Log the result of the function as an output.
* **`log_execution_metrics`**
(`bool`, default:
`False`
)
–Log execution metrics for the task, such as success rate and run count.
* **`tags`**
(`Sequence[str] | None`, default:
`None`
Expand All @@ -2031,9 +1997,9 @@ def task(
scorers: t.Sequence[Scorer[t.Any] | ScorerCallable[t.Any]] | None = None,
name: str | None = None,
label: str | None = None,
log_params: t.Sequence[str] | bool = False,
log_inputs: t.Sequence[str] | bool | Inherited = INHERITED,
log_output: bool | Inherited = INHERITED,
log_execution_metrics: bool = False,
tags: t.Sequence[str] | None = None,
**attributes: t.Any,
) -> TaskDecorator:
Expand All @@ -2054,9 +2020,9 @@ def task(
of the task and will be passed the task's output.
name: The name of the task.
label: The label of the task - useful for filtering in the UI.
log_params: Whether to log all, or specific, incoming arguments to the function as parameters.
log_inputs: Whether to log all, or specific, incoming arguments to the function as inputs.
log_output: Whether to log the result of the function as an output.
log_inputs: Log all, or specific, incoming arguments to the function as inputs.
log_output: Log the result of the function as an output.
log_execution_metrics: Log execution metrics for the task, such as success rate and run count.
tags: A list of tags to attach to the task span.
**attributes: A dictionary of attributes to attach to the task span.

Expand Down Expand Up @@ -2109,9 +2075,9 @@ def task(
for scorer in scorers or []
],
tags=list(tags or []),
log_params=log_params,
log_inputs=log_inputs,
log_output=log_output,
log_execution_metrics=log_execution_metrics,
label=_label,
)

Expand All @@ -2128,7 +2094,6 @@ task_span(
name: str,
*,
label: str | None = None,
params: AnyDict | None = None,
tags: Sequence[str] | None = None,
**attributes: Any,
) -> TaskSpan[t.Any]
Expand All @@ -2150,7 +2115,6 @@ async with dreadnode.task_span("my_task") as task:
Args:
name: The name of the task.
label: The label of the task - useful for filtering in the UI.
params: A dictionary of parameters to attach to the task span.
tags: A list of tags to attach to the task span.
\*\*attributes: A dictionary of attributes to attach to the task span.

Expand All @@ -2166,7 +2130,6 @@ def task_span(
name: str,
*,
label: str | None = None,
params: AnyDict | None = None,
tags: t.Sequence[str] | None = None,
**attributes: t.Any,
) -> TaskSpan[t.Any]:
Expand All @@ -2185,7 +2148,6 @@ def task_span(
Args:
name: The name of the task.
label: The label of the task - useful for filtering in the UI.
params: A dictionary of parameters to attach to the task span.
tags: A list of tags to attach to the task span.
**attributes: A dictionary of attributes to attach to the task span.

Expand All @@ -2200,7 +2162,6 @@ def task_span(
name=name,
label=label,
attributes=attributes,
params=params,
tags=tags,
run_id=run.run_id,
tracer=self._get_tracer(),
Expand Down
47 changes: 47 additions & 0 deletions docs/sdk/serialization.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,53 @@ title: dreadnode.serialization
::: dreadnode.serialization
*/}

seems\_useful\_to\_serialize
----------------------------

```python
seems_useful_to_serialize(obj: Any) -> bool
```

Checks if the object is likely useful to serialize by attempting to
serialize it and checking if the resulting schema indicates a known type.

**Parameters:**

* **`obj`**
(`Any`)
–The Python object to check.

**Returns:**

* **`bool`** ( `bool`
) –True if the object is likely useful to serialize, False otherwise.

<Accordion title="Source code in dreadnode/serialization.py" icon="code">
```python
def seems_useful_to_serialize(obj: t.Any) -> bool:
"""
Checks if the object is likely useful to serialize by attempting to
serialize it and checking if the resulting schema indicates a known type.

Args:
obj: The Python object to check.

Returns:
bool: True if the object is likely useful to serialize, False otherwise.
"""
if obj is None:
return False

with contextlib.suppress(Exception):
_, schema = _serialize(obj)
return schema.get("x-python-datatype") != "unknown"

return False
```


</Accordion>

serialize
---------

Expand Down
Loading