configurable dream/dedup thresholds + safer defaults

[ajianaz]

Problem

Dream pipeline thresholds are all hardcoded constants, making it impossible for users to tune behavior. Current hardcoded values caused aggressive dedup — 17,797 memories reduced to 5,444 (69.4% deleted/deprecated) in a single dream cycle.

Hardcoded Constants (current)

Constant	Value	Location	Problem
SIMILARITY_THRESHOLD (contradict)	0.6	dream.rs:345	Tags + low cosine → contradiction edge. Reasonable, but not tunable.
TAG_JACCARD_MIN (contradict)	0.3	dream.rs:347	30% tag overlap is aggressive for contradiction.
MAX_MEMORIES (contradict scan)	200	dream.rs:348	Limits O(n²) scan, fine.
Consolidate threshold (dedup)	0.92	dream.rs:305	Reasonable — only merges near-identical.
Orphan THRESHOLD	0.3	dream.rs:540	Marks importance < 0.3 as orphan candidates. Too low — most auto-extracted facts sit at ~0.0001 (default importance).
auto_aging_interval_hours	6	config.rs:297	Too frequent.
auto_dream_interval_days	3	config.rs:302	Too frequent for unattended operation.

Root Cause

The orphan phase (THRESHOLD: 0.3) and auto-aging at 6h intervals are the main culprits:

• Auto-extracted memories get default importance ~0.0001 (no explicit boost)
• Orphan scan flags ALL low-importance unpinned memories as candidates
• Combined with auto-aging running every 6h, this creates a compounding cleanup loop

Proposed Changes

1. Add [dream] config section

[dream]
# Contradiction scan
contradict_similarity_threshold = 0.6   # cosine > this → NOT contradict
contradict_tag_jaccard_min = 0.4       # tag overlap ≥ this → scan (up from 0.3)
contradict_max_memories = 200          # O(n²) scan limit

# Dedup (consolidate)
dedup_threshold = 0.92                 # cosine > this → merge candidate

# Orphan detection
orphan_importance_threshold = 0.15    # importance < this → orphan candidate (up from 0.3)
orphan_ttl_days = 90                   # only flag if older than this

2. Safer [maintenance] defaults

[maintenance]
auto_aging_enabled = false              # opt-in, not opt-out
auto_aging_interval_hours = 24          # daily, not every 6h
auto_dream_enabled = true               # keep enabled but...
auto_dream_interval_days = 7           # weekly, not every 3d

3. Safer [aging] defaults

[aging]
enabled = false                         # opt-in
max_age_days = 365                      # 1 year
max_access_count = 10                   # cold = accessed < 10x
max_cold_count = 5000                   # higher threshold before cleanup triggers

Rationale for Default Values

Setting	Old	New	Why
auto_aging_enabled	true	false	Auto-delete should be opt-in. Users losing data without explicit action is bad.
auto_aging_interval_hours	6	24	6h = 4 cycles/day. Even with the 100/cycle safety limit, that's 400/day. Too much.
auto_dream_interval_days	3	7	Weekly is enough for dedup/orphan detection. 3 days is too aggressive for unattended.
orphan_importance_threshold	0.3	0.15	0.3 flags almost everything auto-extracted (default importance ~0.0001). 0.15 is more selective.
orphan_ttl_days	N/A	90	NEW: only flag memories older than 90 days as orphans. Fresh memories should never be flagged.
contradict_tag_jaccard_min	0.3	0.4	Slightly more conservative for contradiction edges. 30% tag overlap is too common.

Implementation Notes

• Config struct: add DreamConfig to config.rs, wire into MaintenanceConfig or standalone [dream] section
• Constants in dream.rs should read from config, falling back to defaults
• Backward compatible: all new config fields have defaults matching proposed safer values

[ajianaz]

✅ Resolved via PR #742

All hardcoded dream pipeline constants replaced with configurable values.

Changes

• [dream] config section with 5 thresholds
• [maintenance] defaults: aging opt-in, intervals 24h/7d
• DreamConfig struct + setter on Uteke
• Server and CLI wired to read config

Breaking (defaults only)

Setting	Old	New
auto_aging_enabled	true	false
auto_aging_interval_hours	6	24
auto_dream_interval_days	3	7
orphan_importance_threshold	0.3	0.15
contradict_tag_jaccard_min	0.3	0.4