Skip to content

Finish longform filter: section → translate → stitch (close TODO in filter.py) #58

Description

@telivity-otaip

Why

There is an explicit TODO in the filter path:

    # TODO(longform): align article inference with translator train sections —
    # section → translate each → stitch via split_translator_source_sections +
    # stitch_filter_chunks. Avoid a half-built whole-article echo path.

Today long drafts trip should_chunk_filter but the section→translate→stitch path is incomplete. Closing this makes filter useful for article-length rewrite (LinkedIn articles, newsletters) instead of echoing whole pieces.

Goal

Implement the longform filter path that:

  1. Splits source into translator-compatible sections
  2. Filters/translates each section with the local adapter/base path
  3. Stitches with stitch_filter_chunks (or equivalent)
  4. Preserves voice guards / force mode semantics

Acceptance criteria

  • Article-length Contoso fixtures filter section-wise, not as one giant echo
  • Stitched output has no duplicated section headers / separator chrome
  • Short drafts still use the existing single-shot path
  • Tests cover split + stitch + chunk threshold boundary
  • TODO comment removed once done
  • README / Advanced notes mention longform filter behavior

Pointers

  • filter.pyshould_chunk_filter, chunk helpers
  • Translator split/stitch helpers (see imports/usages around SFT translator sections)
  • tests/test_filter_voice.py

Difficulty

Medium — existing helpers are partially there; needs careful tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions