Skip to content

LLM makes beneficial improvements to existing translations during updates #1

@mmcky

Description

@mmcky

Observed Behavior

When translating updated sections in UPDATE mode, Claude sometimes improves existing Chinese translations even when those parts weren't changed in the English source.

Example

Test Case: Test 09 (Real-world lecture update)

English Changes:

  • Added phrases like ", with updated examples from recent research"
  • Added "(Leontief systems)" to bullet point
  • NO changes to code comments

Chinese Translation:

  • Translated all English changes correctly ✓
  • ALSO updated code comments: 里昂惕夫列昂惕夫 (improved Leontief transliteration)
  • This change was NOT in the English diff but improves translation accuracy

Why This Happens

In UPDATE mode, Claude receives:

  1. Old English content
  2. New English content (with "Leontief systems" added)
  3. Current Chinese content (with old transliteration 里昂惕夫)

When translating, Claude sees the context and improves terminology consistency.

Assessment

Pros:

  • ✅ Improves translation accuracy
  • ✅ Maintains consistency within document
  • ✅ Uses domain-specific knowledge
  • ✅ Natural behavior for translation assistant

Cons:

  • ⚠️ Changes content not in source diff
  • ⚠️ Could surprise users
  • ⚠️ Could introduce errors if LLM wrong

Recommendation

Currently accepting as beneficial. If problematic later:

Option 1: Add allow_improvements: true flag (default: true)
Option 2: Stricter UPDATE mode prompt
Option 3: Document and monitor (current approach)

Status

  • Current: No action, monitoring
  • Priority: Low
  • Version: v0.4.5+

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions