Skip to content

Externalize memory configuration & add preset Memory Modes (Compact / Balanced / Detail) #571

Description

@haiphucnguyen

TokenAwareSummarizingMemory currently hard-codes all tuning parameters as companion object constants and the constructor receives only a subset of them from ChatConfig. Advanced users have no way to tune memory behaviour without editing config YAML by hand, and there is no concept of preset quality/cost trade-off modes.

Current state

Hard-coded constants in TokenAwareSummarizingMemory (not in any config)

Already in ChatConfig (but not surfaced in UI):

data class ChatConfig(
    val summarizationThreshold: Double = 0.75,
    val enableAsyncSummarization: Boolean = true,
    val summarizationTimeoutSeconds: Long = 300,
    ...
)

The memory budget allocation (40% of context window) is also hard-coded in calculateMaxTokensFromCurrentModel().

Proposed changes

  1. Add Memory Mode
enum class MemoryMode {
    COMPACT,    // Aggressive summarization – lower token cost, less detail
    BALANCED,   // Current defaults – good quality/cost trade-off
    DETAIL      // Minimal summarization – high fidelity, more tokens consumed
}
  1. Define MemoryConfig with per-mode presets
data class MemoryConfig(
    val mode: MemoryMode = MemoryMode.BALANCED,
    // --- overrides (null = use mode default) ---
    val summarizationThreshold: Double? = null,
    val protectedRecentTurns: Int? = null,
    val summarizationPruneFraction: Double? = null,
    val maxKeyFacts: Int? = null,
    val maxMainTopics: Int? = null,
    val maxSummaryLength: Int? = null,
    val memoryBudgetFraction: Double? = null,  // default 0.40
) {
    companion object {
        val COMPACT  = MemoryConfig(mode = MemoryMode.COMPACT,
            summarizationThreshold = 0.25, protectedRecentTurns = 3,
            summarizationPruneFraction = 0.80, maxKeyFacts = 15, maxMainTopics = 8,
            maxSummaryLength = 1000, memoryBudgetFraction = 0.30)
        val BALANCED = MemoryConfig(mode = MemoryMode.BALANCED)   // current defaults
        val DETAIL   = MemoryConfig(mode = MemoryMode.DETAIL,
            summarizationThreshold = 0.60, protectedRecentTurns = 10,
            summarizationPruneFraction = 0.50, maxKeyFacts = 50, maxMainTopics = 25,
            maxSummaryLength = 4000, memoryBudgetFraction = 0.50)
    }
    fun resolve(): ResolvedMemoryConfig = ...   // merge mode defaults + overrides
}
  1. Add memory: MemoryConfig to AppConfigData
data class AppConfigData(
    ...
    val chat: ChatConfig = ChatConfig(),
    val memory: MemoryConfig = MemoryConfig(),   // NEW
)
  1. Thread resolved config into TokenAwareSummarizingMemory
    Replace all companion-object constants with values from ResolvedMemoryConfig passed at construction time (or read from AppConfig.memory).

  2. Surface in AdvancedSettingsSection

Add a Memory Settings card between the existing RAG and Models sections in advancedSettingsSection():
Mode selector: Compact / Balanced / Detail (radio / segmented button)
Expandable "Advanced overrides" panel (hidden by default) exposing each numeric parameter as an editable field — consistent with the existing pattern used in ragConfigurationSection() / modelsConfigurationSection()

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions