Rewards written

[McClain-Thiel]

Note

Introduce bioinformatics-based reward system with evaluation/sweeps, add GRPO runner updates, and wire up CI tests.

• Rewards & Evaluation:
  • Add bioinformatics reward system (src/rewards/bioinformatics/{scorer.py,reward_config.py,logger.py}) and simplify src/rewards/__init__.py.
  • Introduce evaluation pipeline (src/eval/{eval.py,eval_config.py}) and helper utilities (src/utils/training_utils.py).
  • Remove legacy reward files and VERL configs (src/rewards/{rewards.py,plasmid_informatics.py,verl_reward.py}; config/{verl_ppo.yaml,verl_grpo.yaml,verl_naive_ppo.yaml,naive_grpo.yaml}).
• Training:
  • Overhaul GRPO runner (src/runners/grpo.py) to use new scorer, W&B component logging, in-run evaluation, and S3 checkpointing.
  • Add W&B sweep runner (src/runners/grpo_sweep.py).
• Sweeps:
  • Add sweeps/ with configs (configs/*.yaml), docs (README.md, SWEEPS.md), and agent wrapper (run_sweep_agent.py).
• CI & Tests:
  • Add GitHub Actions pytest workflow (.github/workflows/tests.yml).
  • Add unit tests for scorer with sample FASTA data (tests/**); include pytest dependency.
• Config & Infra:
  • Expand src/config.py (datasets, checkpoints path, GRPO hyperparams, W&B defaults).
  • Update docker-compose.yaml (W&B envs, add grpo-sweep service, set WANDB_DIR).
• Docs & Misc:
  • Update README.md with tests badge; extend AGENTS.md with layout and sweep usage.
  • Add example notebook (notebooks/ray_test.ipynb).

^{Written by Cursor Bugbot for commit 441e498. This will update automatically on new commits. Configure here.}

[cursor]

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on December 22

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[cursor]

Bug: Missing Tokenizer in Config, Causes AttributeError

References config.tokenizer which doesn't exist in the Config class. The Config class only defines model but not tokenizer, causing an AttributeError at runtime when this code executes.

 

[cursor]

Bug: Missing config attribute causes runtime AttributeError

References config.storage_bucket which doesn't exist in the Config class. The config defines s3_bucket instead, causing an AttributeError at runtime.

 

[cursor]

Bug: Module datetime misused; now() attribute missing

Calls datetime.now() but only imports the datetime module, not the datetime class. Should be datetime.datetime.now() or import from datetime import datetime. This will raise AttributeError: module 'datetime' has no attribute 'now'.