You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tracks capability gap #3 from the refactor audit: the post-refactor Intervene step applies at every forward step with no way to scope the intervention to a phase of generation (prompt vs generated tokens, or a specific token index).
Supersedes #27 (@akatief's original pre-refactor ask against the old Location.token_pos API): closed as not-planned-in-that-form. This issue tracks the same intent against the current Pipeline + Step architecture.
Requirement
Let callers scope an Intervene step to:
phase="prompt": only during prompt processing
phase="generation": only on generated tokens
phase="all": every step (current behaviour, default)
phase=<int>: a specific generation step index
Core use case: activation patching, where you intervene only during the prompt pass and read the downstream effect on generation.
Design notes
The Intervene step should stay agnostic of how phases resolve to concrete indices. A new phase keyword must not require changes to Step internals: add phases by extending the resolver, not the Step.
The phase→indices resolver should always return a concrete List[int], never slice. Concrete lists make logged token positions readable and remove slice/list branching at call sites. Design input from closed PR Refactor/37 list location resolution #43 (@adham137).
Porting sources
Commit fb766b4 (fix(model): scope interventions to prompt processing). Reference only: the API surface has changed.
Tracks capability gap #3 from the refactor audit: the post-refactor
Intervenestep applies at every forward step with no way to scope the intervention to a phase of generation (prompt vs generated tokens, or a specific token index).Supersedes #27 (@akatief's original pre-refactor ask against the old
Location.token_posAPI): closed as not-planned-in-that-form. This issue tracks the same intent against the currentPipeline+Steparchitecture.Requirement
Let callers scope an
Intervenestep to:phase="prompt": only during prompt processingphase="generation": only on generated tokensphase="all": every step (current behaviour, default)phase=<int>: a specific generation step indexCore use case: activation patching, where you intervene only during the prompt pass and read the downstream effect on generation.
Design notes
Intervenestep should stay agnostic of how phases resolve to concrete indices. A new phase keyword must not require changes to Step internals: add phases by extending the resolver, not the Step.List[int], neverslice. Concrete lists make logged token positions readable and remove slice/list branching at call sites. Design input from closed PR Refactor/37 list location resolution #43 (@adham137).Porting sources
fb766b4(fix(model): scope interventions to prompt processing). Reference only: the API surface has changed.List[int]-over-slicedesign point above.0.1.0-blocking: one of the 5 capability gaps the refactor is missing.