From f6ade8a1dde595b0f9e8fafb611254b99b19fc54 Mon Sep 17 00:00:00 2001 From: Scott Florentino <3495150+smflorentino@users.noreply.github.com> Date: Sun, 12 Jul 2026 21:11:40 -0700 Subject: [PATCH 1/2] docs(readme): clarify framing before hero gif Add a brief tagline before the hero gif clarifying that coder_eval evaluates AI coding agents and their skills (for CLI and skill builders), not an "agentic coding" benchmark. Per Slack feedback. Co-Authored-By: Claude Opus 4.8 --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index c7cc5aa0..8e320bb1 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,11 @@ [![CI](https://github.com/UiPath/coder_eval/actions/workflows/pr-checks.yml/badge.svg)](https://github.com/UiPath/coder_eval/actions/workflows/pr-checks.yml) [![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) +A framework for evaluating AI coding agents **and their skills** — built for CLI +and skill builders — with sandboxing, reproducibility, and data-driven analysis. +Not an "agentic coding" benchmark: it measures how effective your CLI and skills +are when used by coding agents. +

coder_eval running the hello_date task: a sandboxed agent writes and runs a script from a YAML task, then the scored result is browsed in evalboard

From 917ea4d16543a3b3b79e2fb4ee2f29a01139021f Mon Sep 17 00:00:00 2001 From: Scott Florentino <3495150+smflorentino@users.noreply.github.com> Date: Sun, 12 Jul 2026 21:12:48 -0700 Subject: [PATCH 2/2] docs(readme): reframe title toward agents & their skills Co-Authored-By: Claude Opus 4.8 --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8e320bb1..5784bdf4 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# coder_eval — evaluate & benchmark AI coding agents +# coder_eval — evaluate AI coding agents & their skills [![PyPI](https://img.shields.io/pypi/v/coder-eval.svg)](https://pypi.org/project/coder-eval/) [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)