feat: Log Many Metrics#69
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new log_metrics API for batch logging of metrics, standardizes naming from MetricDict to MetricsDict, and cleans up related exports and documentation.
- Introduce
log_metricsmethod with overloads for dict and list inputs - Rename old
MetricDictalias for mapping toMetricsDictand addMetricDictTypedDict for individual metric configs - Export
log_metricsin the package root and update all docs to cover the new API
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| dreadnode/tracing/span.py | Updated imports and type hints from MetricDict to MetricsDict |
| dreadnode/metric.py | Added MetricDict TypedDict and renamed mapping alias to MetricsDict |
| dreadnode/main.py | Added log_metrics implementation, updated log_metric rename, expanded docstring |
| dreadnode/init.py | Export log_metrics (and log_outputs) at package root |
| docs/usage/metrics.mdx | Updated usage example to show log_metrics and added a Tip |
| docs/sdk/metric.mdx | Documented new MetricsDict and ScorerResult; added MetricDict section |
| docs/sdk/main.mdx | Added detailed API reference and examples for log_metrics |
Comments suppressed due to low confidence (5)
dreadnode/metric.py:22
- [nitpick] The name
MetricDictfor a TypedDict may be confused with the pluralMetricsDict; consider renaming toMetricConfigor similar to clarify its purpose.
class MetricDict(te.TypedDict, total=False):
dreadnode/main.py:940
- You’ve changed the parameter from
keytonamewhich is a breaking API change; consider providing a deprecated alias forkeyto ease client upgrades.
+ name: str,
docs/sdk/metric.mdx:221
- The
MetricDictsection lacks a code snippet showing its fields or usage; adding a proper example or the TypedDict definition would improve clarity.
+MetricDict
dreadnode/main.py:1090
- There are no tests covering the new
log_metricsmethod; adding unit tests for both dict and list inputs will ensure it behaves as expected.
def log_metrics(
dreadnode/init.py:29
- Ensure that
log_outputsis actually implemented onDEFAULT_INSTANCEor remove this export to prevent an AttributeError at runtime.
log_outputs = DEFAULT_INSTANCE.log_outputs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
log_metricsfunction.Generated Summary:
log_metricandlog_metricsfunctions for improved metrics logging.log_metricsupports additional parameters:Metricobjects for flexibility.log_metricsoffers the ability to log multiple metrics at once via dictionary or list of metric configurations.MetricDictandMetricsDictfor structured metric representation, enhancing API usability.keytonamein relevant functions.This enhancement allows users to easily log and manage metrics, improving tracking of performance and resource usage during task execution.
This summary was generated with ❤️ by rigging