fix(fugu): accept named workflow lists in any order - #467
Open
tryeverything24 wants to merge 1 commit into
Open
Conversation
The Conductor keys its three workflow lists by name (model_id/subtasks/access_list), but the parse-gate additionally required them in canonical textual order, so a valid proposal emitting them in another order returned (None, False) -> parsed_ok=False -> scored 0, corrupting the GRPO reward. Keep the ordered search as a first pass (unchanged), then add an order-independent second pass that runs only when the first finds nothing and skips triples already tried (m_pos<=s_pos<=a_pos). All validation gates still apply in both passes, so malformed proposals still fail. The ordered pass returns on first success, so the change can only ADD acceptance for previously-rejected valid inputs -- provably non-regressive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(fugu): accept named workflow lists in any order
Summary
The Conductor proposes a workflow as three parallel lists, each keyed by name
(
model_id/subtasks/access_list). The parse-gate inparse_workflow(src/trinity/fugu/workflow.py) already disambiguates thethree lists by name, but the candidate-pairing loop additionally required
them to appear in that exact textual order:
A perfectly well-formed proposal that emits the three correctly-named lists in
a non-canonical order (e.g.
access_listbeforesubtasks) therefore fellthrough to
return None, False. That surfaces asparsed_ok=False, so thereward layer scores the workflow 0 even though the proposal was valid —
directly corrupting the GRPO training reward signal. The ordering constraint
was never semantically needed, since the lists are already keyed by name.
Fix
Keep the ordered search as a first pass (unchanged), then add an
order-independent second pass that runs only when the first pass finds
nothing and inspects only the triples the first pass skipped (non-canonical
order):
model_id, thensubtasks, thenaccess_list). Preserves every existing pairing, and still lets ascratch/reasoning list that precedes the real block be skipped over.
triples, guarded by
if m_pos <= s_pos <= a_pos: continueso it neverre-examines a triple the ordered pass already handled.
Every gate in
_validate_workflow_listsstill applies in both passes, somalformed proposals (missing/extra list, length mismatch, non-int or
out-of-range
model_id, forward reference, empty subtask, over-lengthworkflow) still fail exactly as before.
Provably non-regressive: the ordered pass runs first and returns on the
first success, so control only reaches the second pass when no canonical
ordering validated. The second pass explicitly skips canonical-order triples.
It can therefore only add acceptance for previously-rejected valid inputs —
it can never change or lose an existing pairing.
Tests
Added
test_parse_accepts_named_lists_in_noncanonical_order(mirrors theexisting style in
tests/test_fugu_workflow.py): a valid proposal with thethree named lists in a non-canonical order (
access_listbeforesubtasks)now parses successfully and yields the correct steps. Red before the fix,
green after.
Local verification (venv,
PYTHONPATH=src):ruff check— cleanmypy src/trinity/fugu/workflow.py— cleanpytest tests/test_fugu_workflow.py -q— 9 passed (8 pre-existing + 1 new)The full-suite failures in
tests/test_torch_optim_sft.pyare pre-existing onorigin/mainand unrelated to this change (a different module that does notimport
trinity.fugu.workflow); this change introduces no new failures.