Skip to content

Eliminate Snapshot Duplication in Session JSON #130

@CyberSecDef

Description

@CyberSecDef

Problem

The session JSON file stores every piece of novel data twice:

  1. Top-level keystitle, premise, genre, chapters, etc. (~199 KB)
  2. progress_data.snapshot — An identical copy (~199 KB)

Two independent persistence paths evolved:

  • save_session_state() writes top-level keys
  • persist_completed_chapters() writes full progress_manager.get(token) dict (including snapshot)

Duplication doubles file size (~400 KB before chapters, ~1-2 MB for 25-chapter novel).

Files Affected

  • novelforge/routes/generation/chapters.py lines 67-102
  • novelforge/routes/generation/revision.py lines 78-92
  • novelforge/routes/export.py lines 113, 138, 555-603
  • novelforge/session/persistence.py lines 226-280, 329-351, 354-407, 471-512
  • novelforge/progress.py

Why It Matters

  • File size is 2x what it needs to be
  • Two copies can drift if a bug updates one but not the other
  • ~200 KB redundant data through read-modify-write cycle on every chapter completion

Recommended Approach

  1. Stop persisting the snapshot — strip snapshot from progress data before writing
  2. Rebuild snapshot on load from top-level keys
  3. Eliminate completed_chapters / chapters_done duplication
  4. Ensure every stage change persists
  5. Backward compatibility for old session files

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:criticalCritical priority

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions