benchmarks: rlpr / ifeval / bfcl_simple load() all raise TypeError (same bug as #86, masked by the same test pattern)

[philluiz2323]

Bug

The three new benchmark facades — benchmarks/rlpr.py, benchmarks/ifeval.py, benchmarks/bfcl_simple.py (added in #104, #103, #102) — each raise TypeError on every call to their public load(). This is the identical defect that benchmarks/livecodebench.py had and that was fixed + regression-tested in #90; the three newer facades regressed the same way.

from trinity.orchestration.dataset import load_tasks as _load_tasks

def load(split, *, max_items=None, seed=0, allow_toy_fallback=False):
    return load_tasks(          # <-- the SIBLING load_tasks, not _load_tasks
        "rlpr",
        split,
        max_items=max_items,
        seed=seed,
    )

def load_tasks(split, *, max_items=None, seed=0, allow_toy_fallback=False):
    return _load_tasks("rlpr", split, max_items=max_items, seed=seed)

load() passes two positionals ("rlpr", split), but the sibling load_tasks(split, *, ...) accepts one. The benchmark-name positional belongs to the imported _load_tasks(benchmark, split, ...).

Reproduce (offline, no network — the crash precedes any loading)

>>> import benchmarks.rlpr, benchmarks.ifeval, benchmarks.bfcl_simple
>>> benchmarks.rlpr.load("test", max_items=1)
TypeError: load_tasks() takes 1 positional argument but 2 positional arguments (and 2 keyword-only arguments) were given

Same for benchmarks.ifeval.load(...) and benchmarks.bfcl_simple.load(...). benchmarks.livecodebench.load(...) (fixed in #90) does not raise.

Impact

load() is the documented config-facing entrypoint. benchmarks/__init__.py states: "Each loader exposes load(split, **kw) -> list[Task]", and configs/benchmarks.yaml names all three modules as loader: for the OOD suite (ifeval, rlpr, bfcl_simple). Any caller going through the documented facade — rather than reaching past it to load_tasks / dataset.load_tasks directly — cannot load these three benchmarks at all. This is the same public-API break that #90 fixed for LiveCodeBench.

Why CI stayed green

Each facade test masks the bug exactly as the pre-#90 livecodebench test did — it monkeypatches the sibling load_tasks (the very function whose signature is the bug) with a four-positional stub, so the two-positional miscall type-checks against the fake:

def fake_load_tasks(benchmark, split, max_items, seed):   # four positionals
    ...
monkeypatch.setattr(RLPR, "load_tasks", fake_load_tasks)

A stub whose signature does not match the function it replaces cannot catch a signature bug. The three new facades never received the load_is_callable regression test that #90 added.

Expected

Mirror the merged #90 fix: load() should delegate to the sibling wrapper, naming the benchmark in one place —

def load(split, *, max_items=None, seed=0, allow_toy_fallback=False):
    return load_tasks(split, max_items=max_items, seed=seed)

— and each facade test should patch _load_tasks (the real delegation boundary) plus add a test that calls load() without patching load_tasks.

Location

benchmarks/rlpr.py:21, benchmarks/ifeval.py:21, benchmarks/bfcl_simple.py:21. Masking tests: tests/test_rlpr.py, tests/test_ifeval.py, tests/test_bfcl.py.