Add public mock discord runtime#21
Conversation
bingran-you
force-pushed
the
bry/public-mock-discord-runtime
branch
from
3100944 to
91faf8d
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 310094419a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - mock-gdoc | ||
| - mock-gdrive | ||
| - mock-slack | ||
| - mock-discord |
There was a problem hiding this comment.
Adding mock-discord to this matrix makes the existing conformance step run uv run --extra dev pytest tests/test_conformance.py -q in packages/environments/mock-discord, but that package only adds tests/test_discord_conformance.py and no tests/test_conformance.py (confirmed with repo-wide search). As a result, every push/PR CI run for this matrix entry fails before collecting tests unless the file is renamed, a wrapper is added, or the workflow is special-cased.
Useful? React with 👍 / 👎.
| engine = get_engine() | ||
| Base.metadata.drop_all(engine) | ||
| try: | ||
| result = seed_database(scenario=scenario, seed=seed) |
There was a problem hiding this comment.
When the server is launched with the manifest/task path (--db /data/discord.db), calling /_admin/seed drops the active /data/discord.db engine and then calls seed_database() without passing that path, so seed_database resets the engine and recreates the default .data/mock_discord.db instead. This leaves the verifier DB at /data/discord.db empty/dropped after an admin reseed; capture the current engine URL and pass it as db_path, as the other services do.
Useful? React with 👍 / 👎.
| [mock-discord] | ||
| port = 9006 | ||
| db_path = "/data/discord.db" | ||
| env_var = "DISCORD_URL" |
There was a problem hiding this comment.
/workspace/env0/AGENTS.md says to prefer current mock-* / MOCK_* names and not add legacy service-name contracts. Registering the new service as DISCORD_URL makes the launcher/base image expose a new legacy-style variable (and derives DISCORD_DB_PATH), so the new task prompt and Docker env codify the inconsistent contract instead of the expected MOCK_DISCORD_URL/MOCK_DISCORD_DB_PATH naming.
Useful? React with 👍 / 👎.
1 participant
Summary
mock-discordenvironment package with CLI/runtime docs, conformance fixtures, and testsdiscord-incident-followupunder bothexample_tasks/andtasks/with oracle and verifier coverageValidation
uv run --extra dev pytest tests -qinpackages/environments/mock-discord(139 passed)uv run --extra dev pytest tests/test_conformance.py -qinpackages/environments/mock-discord(32 passed, after fixture redaction)python3 -m unittest tests/test_env0_control.py(23 tests OK)python3 scripts/env0_control.py seed-task discord-incident-followup mock-discordmock-discordserve + oracle + verifier (reward: 1.0, no wrong-channel messages)scripts/smoke_dev.shdocker/build-base.sh && PULL_BASE=0 scripts/smoke_docker_examples.shBENCHFLOW_REWARD_LENIENT=1 bench eval run --tasks-dir example_tasks --include discord-incident-followup --agent oracle --sandbox docker --jobs-dir .local/bf-jobs-public-discord-example-oracle --context-root . --concurrency 1 --build-concurrency 1(1/1, reward1.0)BENCHFLOW_REWARD_LENIENT=1 bench eval run --tasks-dir example_tasks --agent oracle --sandbox docker --jobs-dir .local/bf-jobs-public-examples-oracle-discord --context-root . --concurrency 1 --build-concurrency 1(8/8, score100.0%)git diff --check