From 98fa1d1afa7320685e85a74e4aefe0b73d36db96 Mon Sep 17 00:00:00 2001
From: Md Farhan Ishmam <45528856+farhanishmam@users.noreply.github.com>
Date: Sun, 19 Jul 2026 21:40:51 -0600
Subject: [PATCH] Added Knows: a Google Workspace benchmark for web agents
 (Docs, Sheets, Slides)

Knows evaluates browser agents on long-horizon document authoring in Google
Workspace. An agent drives a real browser to build a Doc, Sheet, or Slide deck
from a natural-language goal, and the result is graded checkpoint-by-checkpoint
via the Workspace APIs, yielding a fractional reward rather than binary success.

110 tasks across 22 families and 3 Workspace apps (25 docs / 45 sheets /
40 slides), exposed as a single `knows` split.

This is the AgentLab-side counterpart to the BrowserGym change that adds the
`knows` action subset, benchmark config, and task metadata. As with TimeWarp,
the benchmark itself lives in an external package (`browsergym-knows`), so
AgentLab needs only the lazy import and a docs entry:

- lazy `import browsergym.knows` in `_get_env_name`, so worker processes
  register the gym envs before `gym.make` (joblib/ray workers get a fresh
  `sys.modules`, so the parent's `prepare_backends()` import does not carry
  over)
- a `Knows` row in the supported-benchmarks table

No dependency change, matching the TimeWarp precedent: AgentLab declares no
per-benchmark extras, and benchmark packages are installed out of band per the
setup link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 README.md                        | 1 +
 src/agentlab/experiments/loop.py | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/README.md b/README.md
index 40611b4d..0f4c0e18 100644
--- a/README.md
+++ b/README.md
@@ -64,6 +64,7 @@ AgentLab Features:
 | [MiniWoB](https://miniwob.farama.org/index.html) | [setup](https://github.com/ServiceNow/BrowserGym/blob/main/browsergym/miniwob/README.md) | 125 | Medium | 10 | no | self hosted (static files) | soon |
 | [OSWorld](https://os-world.github.io/) | [setup](https://github.com/ServiceNow/AgentLab/blob/main/src/agentlab/benchmarks/osworld.md) | 369 | None | - | - | self hosted  | soon |
 | [TimeWarp](https://timewarp-web.github.io/) | [setup](https://github.com/sparklabutah/timewarp) | 1386 | None | 30 | yes | self hosted | soon |
+| [Knows](https://github.com/alexgill321/Agent-Benchmark) | [setup](https://github.com/alexgill321/Agent-Benchmark) | 110 | None | 120 | yes | live web | soon |
 
 
 ## 🛠️ Setup AgentLab
diff --git a/src/agentlab/experiments/loop.py b/src/agentlab/experiments/loop.py
index 486e2f35..b3d409b6 100644
--- a/src/agentlab/experiments/loop.py
+++ b/src/agentlab/experiments/loop.py
@@ -928,6 +928,8 @@ def _get_env_name(task_name: str):
         import weblinx_browsergym
     elif task_name.startswith("timewarp"):
         import browsergym.timewarp
+    elif task_name.startswith("knows"):
+        import browsergym.knows
 
     return f"browsergym/{task_name}"