Skip to content

feat(find_symbol): report searched kinds, index module/class variables#68

Closed
andrebrait wants to merge 353 commits into
Mibayy:mainfrom
andrebrait:feat/find-symbol-kinds
Closed

feat(find_symbol): report searched kinds, index module/class variables#68
andrebrait wants to merge 353 commits into
Mibayy:mainfrom
andrebrait:feat/find-symbol-kinds

Conversation

@andrebrait

Copy link
Copy Markdown
Contributor

Problem

find_symbol searches functions and classes — and only those. _resolve_symbol_info reads meta.functions and meta.classes on both its symbol-table fast path and its all-files fallback, and StructuralMetadata has no field for anything else. That has been true since the first commit; there is no variables field to consult.

The failure mode is not the missing coverage, it is what a miss claims:

{"error": "symbol 'pfb' not found", "scanned_files": 2085, "complete": true}

plus a docstring promising callers they "do not need to fall back to grep/search_codebase". pfb in that project is a module-level dict referenced 317 times in a single file. An agent that trusts complete: true concludes the symbol does not exist and stops looking.

Changes

1. Honest miss reporting. A miss now returns the kinds it actually searched, and — when an indexed kind was excluded — what it skipped plus the exact retry:

{
  "error": "symbol 'pfb' not found",
  "scanned_files": 2085,
  "searched": ["function", "class"],
  "not_searched": ["variable"],
  "retry_with": "find_symbol(name='pfb', kinds=['variable'])"
}

complete: true stays on the hit path, where it is accurate. The MCP _suggestion leads with the retry instead of sending the caller straight to grep.

2. kinds parameter and variable indexing. find_symbol(name, kinds=[...]) takes function, class, and variable. Variables cover module globals, constants (UPPER_SNAKE or a Final[...] annotation), and class attributes. The Python annotator extracts them from Assign / AnnAssign at module and class scope. Function locals are skipped — they cannot be referenced from another module, so indexing them would grow the index without making anything findable. AugAssign is skipped as well: x += 1 rebinds an existing name, so treating it as a definition would report the wrong line.

Bindings go in a new ProjectIndex.variable_table (bare name and qualified name → defining files), deliberately not in symbol_table:

  • a module global can never shadow a same-named function in the resolvers that walk symbol_table;
  • same-named globals across files (logger, DEFAULTS) are routine, so the value is a list and a collision reports as ambiguous rather than silently picking a winner.

3. TOKEN_SAVIOR_VARIABLES gates both halves:

value effect
off annotators skip extraction; an explicit kinds=["variable"] is rejected with an explanation, rather than returning an empty result that reads as "no such variable"
index (default) variables are indexed, searched only when requested
search variables are indexed and included in the default kinds

Read per call, so it can be flipped without a server restart.

Compatibility

  • _CACHE_VERSION 2 → 3. The new variables field is persisted; a stale v2 cache would report every project as variable-free.
  • Default kinds is the previous behaviour, so existing callers see no change apart from the richer miss payload.
  • The MCP handler forwards kinds only when the caller sets it, so query engines with the older two-argument find_symbol signature keep working.
  • Annotators other than Python leave variables empty. searched / not_searched report that honestly instead of hiding it. Adding extraction to another annotator is now a per-language change with no further plumbing.

Tests

tests/test_symbol_kinds.py, 24 tests: miss reporting, extraction (module/class scope, constant detection, locals excluded), kinds lookup and isolation, ambiguity, all three env-var modes, cache round-trip, and a guard that the cache version was bumped. Full suite: 1813 passed, 1 pre-existing failure (test_daemon_client.py::test_successful_call_returns_text — pytest's tmp_path exceeds the 104-byte macOS AF_UNIX limit; unrelated to this change and green on the same machine under a shorter path).

ruff check src/ tests/ clean.

One doc gap left deliberately: the new variable is documented in the variables_mode() docstring rather than the README, since the README has no environment-variable reference yet (#56).

joshorig and others added 30 commits April 13, 2026 13:22
…ty pre-warming

- PPM (Prediction by Partial Matching) up to order-5 with escape-probability
  blending over the first-order fallback. Persisted alongside the existing
  Markov table and wired into beam_search_continuations for richer speculation.

- Token Economy ROI: compute_observation_roi uses exponential decay of P(hit)
  times per-type multiplier; run_roi_gc archives negative-ROI observations
  (dry_run by default). memory_roi_gc + memory_roi_stats tools, monthly GC
  via SessionStart hook.

- Leiden community pre-warming: greedy modularity maximization on the symbol
  dependency graph runs after reindex. get_function_source emits a 🏘️
  community footer and _warm_cache_async prefetches community peers when the
  cluster is small. get_community + get_leiden_stats tools.

Tool count: 86 → 90. 891/891 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… abstractions

Rissanen MDL: replace N similar observations with 1 abstraction + N deltas
when L(abstraction) + Σ L(delta) < Σ L(obs).

- mdl_distiller.py: Jaccard agglomerative clustering per type, compact
  abstraction ([MDL:type] core_tokens — representative), delta_encode strips
  shared tokens so each member stores only its novel fragments.
- memory_db.run_mdl_distillation: dry_run preview or apply — creates
  type=convention decay_immune abstraction, rewrites members to
  "[delta] ...\n[abstraction_id: N]", tags 'mdl-distilled', inserts
  'supersedes' observation_links.
- memory_distill tool + ts memory distill CLI.
- SessionEnd hook suggests distillation when >20 undistilled obs have
  candidates.
- get_usage_stats: MDL line (abstractions + distilled count).

Tool count: 90 → 91. 891/891 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PPM variable-order Markov (order 2-5)
Token Economy ROI GC
Leiden community pre-warming
MDL memory distillation
LinUCB contextual bandit injection (10-feature model)
Cross-session warm start (cosine signature matching)
Self-consistency check (Bayesian validity, quarantine)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrates @joshorig's Java support branch (feat/java-language-support)
on top of v2.3.0. Resolves Mibayy#10.

Adds:
- tree-sitter-based java_annotator (classes, interfaces, enums, records,
  constructors, methods, annotations, qualified names)
- gradle_annotator for build files
- Java dependency graph (invocations, constructor edges, method refs,
  lambda paths, interface→implementation, Spring-aware entry edges)
- Java coverage in find_symbol / get_dependencies / get_dependents /
  get_change_impact / get_call_chain / get_routes / get_entry_points /
  find_dead_code / find_impacted_test_files
- 3 new MCP tools: find_allocation_hotspots, find_performance_hotspots,
  get_duplicate_classes (tool total: 95 → 98)

Rebased conflicts resolved:
- tests/test_tool_schemas.py count 89 → 98

Dependencies added (hard):
- tree-sitter >=0.25,<0.26
- tree-sitter-java >=0.23,<0.24

Tests: 1006/1006 pass (891 existing + 115 new Java tests).
Ruff clean on all new files.

Follow-ups tracked separately (per PR Mibayy#11 author's own notes):
- find_performance_hotspots too broad on .get() patterns
- get_class_source(level=2) semantic summary quality

Co-Authored-By: joshorig <joshorig@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Extracted run_migrations() with per-path idempotency (called at MCP
  startup + lazily from get_db as safety net for tests).
- get_db() is now minimal: just opens a WAL connection, no schema work.
- Added @contextmanager db_session() guaranteeing close() on exception.
- Refactored WRITE functions to use db_session(): observation_save,
  session_start, session_end, run_decay, run_roi_gc, run_consistency_check,
  run_mdl_distillation, auto_link_observation, update_consistency_score.
- Migrations now run after executescript (fixes fresh-DB ordering bug:
  ALTER TABLE sessions ADD COLUMN end_type was skipped when sessions
  table didn't yet exist).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add _META_HANDLERS dispatch table with 18 _hm_* handlers (meta tools).
- Extract _track_call, _maybe_compress, _prefetch_next helpers.
- call_tool: 395 → 38 lines; flat handler dispatch (meta/memory/slot/qfn).
- 1006/1006 tests pass, ruff clean.
…on, utils)

- New db_core.py: MEMORY_DB_PATH, run_migrations, get_db, db_session,
  _now_iso, _now_epoch, _json_dumps, observation_hash, strip_private,
  relative_age, _fts5_safe_query. 201 lines, zero higher-level deps.
- memory_db.py re-exports the public surface + thin wrappers for get_db /
  db_session / run_migrations that read memory_db.MEMORY_DB_PATH at call
  time, so test patching continues to work.
- 1006/1006 tests pass, ruff clean.

Step B partial: obs_store / session_store / memory_ops extraction deferred —
those require unwinding tight cross-function coupling across ~80 functions
and 7 importers; acceptable follow-up given current win already isolates
the SQLite plumbing.
Adds 33 tests covering v2.x feature surface that lacked direct
coverage (LinUCB, SessionWarmStart, Consistency, MDL, PPM, TCA,
Leiden). Tool manifest compressed from 51620 to 41531 chars
(-19.5%) by trimming verbose descriptions and parameter hints
on the top verbose tool schemas.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- TOKEN_SAVIOR_STATS_DIR env var overrides ~/.local/share/token-savior
  in server.py, slot_manager.py, query_api.py (dashboard.py already did)
- tests/conftest.py sets TOKEN_SAVIOR_STATS_DIR to a fresh tempdir at
  import time, before any test module loads server/slot_manager — kills
  the 196 orphan test_*.json leak cleanly
- type hints: get_db/db_session/run_migrations wrappers in memory_db.py
  now return sqlite3.Connection / AbstractContextManager[Connection]
- bulk dict/list generic fixes: bare `dict` → `dict[str, Any]`,
  bare `list` → `list[Any]` across memory_db.py + server.py

mypy errors: memory_db 170 → 41 (-76%), server 162 → 122 (-25%)
tests: 1039/1039 passing, ruff clean, 0 prod state pollution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add pip install with [mcp] extra to Quick Start
- Add claude mcp add one-liner
- Fix binary name in YAML example (token-savior-recall -> token-savior)
- Clarify .mcp.json vs settings.json scoping

Closes Mibayy#12

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multi-step refactor sweep — 1069/1069 tests green throughout.

Shared utilities:
- output_helpers.py: extracted _build_line_offsets + _truncate_output
  (used by impacted_tests.py, project_actions.py)
- brace_matcher.py: rewrote find_brace_end_csharp with explicit while-loop
  state; the original `for idx in range(...)` rebound idx every iteration,
  losing the verbatim-string skip and reading } as code (covered by
  characterization tests in tests/test_brace_matcher.py)

server.py — _format_usage_stats:
- 278-line function with depth 6 → 33-line orchestrator + 18 _usage_*
  section helpers (each ≤50 lines, depth ≤3)

query_api.py — create_file_query_functions:
- 247-line factory with 13 nested closures → 26-line dispatcher using
  functools.partial + 13 module-level _file_*_impl helpers
- Public dict shape and callable signatures unchanged

Annotator changes (java/gradle): adopt the new output_helpers utility.
Move all module-level mutable globals from server.py to a new
src/token_savior/server_state.py. Pure relocation, zero behavior change.

server.py now imports state via:
  from token_savior import server_state as s
  from token_savior.server_state import server

All references to moved globals (counters, caches, engine instances,
slot manager, tool-set frozensets) are routed via `s.<name>`. Local
variables that previously shadowed `s` are renamed (`stats`, `summary`,
`sess`, `sug`) to keep the module alias unambiguous.

Tests updated to mutate scalar state through server_state directly,
since rebinding via the server alias would no longer propagate.

1069 tests pass.
Move cross-cutting request-cycle helpers out of server.py:
- _fmt_lines, compress_symbol_output (compact-output formatting)
- _detect_client_name, _parse_workspace_roots, _register_roots
- _get_stats_file, _load_cumulative_stats, _flush_stats
- _format_result, _count_and_wrap_result, _estimate_naive_chars_for_call
- _prep, _warm_cache_async, _recompute_leiden, _resolve_project_root
- _TOOL_COST_MULTIPLIERS, _CLIENT_NAME, _SESSION_LABEL constants

server.py re-exports them so existing call sites and tests stay working.
Drops imports that were only feeding the moved code (Server, SlotManager,
PPMPrefetcher, TCAEngine, LeidenCommunities, LinUCBInjector,
SessionWarmStart, Path, hashlib, threading, uuid).

server.py: 3232 → 2732 lines.
1069 tests pass.
First handler split: move _h_discover_project_actions and
_h_run_project_action to server_handlers/project_actions.py, exported
via a HANDLERS dict that gets spread into the existing _SLOT_HANDLERS
table.

Adds the server_handlers package skeleton (__init__.py) that step 13
will use to aggregate ALL_HANDLERS with collision detection.

1069 tests pass.
Move the four git-flavored handlers (get_git_status,
get_changed_symbols, summarize_patch_by_symbol, build_commit_summary)
to server_handlers/git.py with a HANDLERS dict spread into _SLOT_HANDLERS.

Drops the now-unused git_tracker / git_ops / compact_ops imports from
server.py.

1069 tests pass.
Move four edit-flavored handlers (replace_symbol_source,
insert_near_symbol, verify_edit, apply_symbol_change_and_validate) into
server_handlers/edit.py with a HANDLERS dict spread into _SLOT_HANDLERS.

Drops the now-unused edit_ops / workflow_ops imports from server.py.

1069 tests pass.
Move the six checkpoint handlers (create/list/delete/prune/restore/
compare) into server_handlers/checkpoints.py with a HANDLERS dict
spread into _SLOT_HANDLERS.

Drops the now-unused checkpoint_ops import from server.py.

1069 tests pass.
Move the two test-impact handlers (find_impacted_test_files,
run_impacted_tests) into server_handlers/tests.py with a HANDLERS
dict spread into _SLOT_HANDLERS.

Drops the now-unused impacted_tests import from server.py.

1069 tests pass.
Move the eight code-quality and analysis handlers (analyze_config,
find_dead_code, find_hotspots, find_allocation_hotspots,
find_performance_hotspots, detect_breaking_changes,
find_cross_project_deps, analyze_docker) into
server_handlers/analysis.py with a HANDLERS dict spread into
_SLOT_HANDLERS.

Drops the now-unused breaking_changes/complexity/config_analyzer/
cross_project/dead_code/docker_analyzer/java_quality/ProjectIndex
imports from server.py.

Incidentally fixes a latent shadowing bug in
_h_find_cross_project_deps where the loop variable s shadowed the
module alias s; the new module renames the loop variable to
sibling_slot, mirroring _h_find_dead_code.

1069 tests pass.
Move the four project-lifecycle manager handlers (list_projects,
switch_project, set_project_root, reindex) into
server_handlers/project.py with a HANDLERS dict spread into
_META_HANDLERS.

The new module references state._slot_mgr explicitly instead of
relying on the server.py module alias `s`, matching the pattern of
the other extracted handler modules.

1069 tests pass.
Move the 28-entry _QFN_HANDLERS dispatch table plus the CSC
(Compact Symbol Cache) subsystem (_lookup_symbol_meta,
_csc_compact_response, _csc_diff_preview, _csc_maybe_serve) and the
three named query handlers (_q_get_class_source,
_q_get_function_source, _q_get_edit_context) into
server_handlers/code_nav.py.

server.py imports the dict as QFN_HANDLERS and re-exports
_q_get_edit_context for tests/test_server.py.

Drops the redundant s._PREFETCHABLE_TOOLS reassignment from server.py
since server_state.py already defines the same frozenset.

1069 tests pass.
Move the 18 _usage_* section builders, _format_usage_stats,
_format_duration, and the 12 _hm_get_* meta-handlers
(get_usage_stats, get_session_budget, get_coactive_symbols,
get_tca_stats, get_dcp_stats, get_community, get_linucb_stats,
get_warmstart_stats, get_leiden_stats, get_speculation_stats,
get_lattice_stats, get_call_predictions) into
server_handlers/stats.py.

server.py shrinks from 1860 to 1314 lines. The stats handlers
are spread into _META_HANDLERS via **_STATS_HANDLERS.

_format_duration and _format_usage_stats are re-exported from
server.py so tests/test_usage_stats.py keeps working.

_hm_get_session_budget uses a lazy import for
_resolve_memory_project (still in server.py until step 12).

1069 tests pass.
Move 26 _mh_* memory tool handlers, 2 _hm_* admin handlers
(memory_consistency, memory_quarantine_list), and 4 helpers
(_resolve_memory_project, _invalidate_injection_hash,
_linucb_credit_reward, _build_linucb_context) into
server_handlers/memory.py.

server.py shrinks from 1314 to 260 lines.

stats.py now imports _resolve_memory_project eagerly from
memory.py (no more lazy import workaround).

_resolve_memory_project is re-exported from server.py for
backward compatibility.

1069 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add server_handlers/__init__.py that merges per-domain HANDLERS dicts
into four call-shape categories (META, MEMORY, SLOT, QFN) and asserts
the union is disjoint, so a tool name accidentally added to two
domains fails at import time rather than silently shadowing.

server.py now imports the four aggregated dispatch tables from the
package, eliminating the inline _META_HANDLERS / _SLOT_HANDLERS dict
constructions.

Re-exports of _q_get_edit_context, _resolve_memory_project,
_format_duration, and _format_usage_stats kept for tests.

1069 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Drop unused imports (json, os, time) and unused server_runtime
  re-exports (_CLIENT_NAME, _SESSION_LABEL, _TOOL_COST_MULTIPLIERS,
  _detect_client_name, _estimate_naive_chars_for_call, _fmt_lines,
  _get_stats_file, _recompute_leiden, _resolve_project_root) that
  no caller imports anymore.
- Drop _resolve_memory_project re-export (only used internally).
- Restore missing 'import sys' that was accidentally removed during
  the server_runtime extraction; the exception handler in call_tool
  references sys.stderr but no caller had hit that path in tests so
  the broken import sat dormant.
- Sort imports and drop the now-stale section-comment headers.

server.py: 230 -> 202 lines.

1069 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ffset bug

python_annotator carried its own _compute_line_offsets helper that called
source.splitlines(keepends=False) and then incremented offsets by len(line)+1.
On CRLF input that arithmetic is off by one per line: splitlines() strips the
trailing \r so the +1 only accounts for the \n, leaving every line offset 1
byte short cumulatively.

Drop the private helper, import build_line_char_offsets from models, and split
with source.split('\n') -- the convention used by ~16 other annotators. That
preserves the trailing \r inside each line so len(line)+1 correctly accounts
for the full \r\n pair.

Verified on CRLF input 'x = 1\r\ny = 2\r\nz = 3\r\n' (21 chars):
  before: offsets = [0, 6, 12]  (y at 6, actual position 7 -- wrong)
  after:  offsets = [0, 7, 14, 21]  (matches str.index lookups exactly)

Two test_markup_python edge-case tests asserted the legacy splitlines() line
count for empty source (0) and trailing-newline source (2). With split('\n')
those become 1 and 3 respectively, which now matches every other annotator's
total_lines convention. Tests updated with explanatory comments.

1069 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the 90-line monolithic `find_brace_end_rust` into a 36-line driver
plus four focused helpers, mirroring the csharp refactor pattern. Also
fixes a latent `for idx in range` rebind bug where multi-line raw-string
advancement was lost when control returned to the outer loop (the new
driver uses a single `while idx < len(lines)` loop).

Helpers extracted:
- _rust_skip_block_comment        — handles nested /* ... */
- _rust_try_skip_raw_string       — r#..."..."#... with EOF handling
- _rust_skip_string               — regular "..." with backslash escapes
- _rust_skip_char_or_lifetime     — 'a' / '\n' vs lifetime 'a

LOC:   90 → 36 (driver) + 4 helpers (~85 LOC total, including docstrings)
Depth: ~5 → ~3 (driver), ≤3 in helpers
Tests: +26 characterization cases (TestFindBraceEndRust), all green;
       full brace_matcher suite 56/56 passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hermes and others added 26 commits May 19, 2026 10:10
…eholder

v4.3.0 ts init was broken on vanilla pip install: hooks/ was missing from the wheel
and JSON configs had hard-coded paths to the dev machine. v4.3.1 fixes both via
hatch force-include + placeholder substitution in _load_hook_bundles().
v3.5.0..v4.3.2: every successful tool call returned isError=True
with a CallToolResult pydantic validation error. The shim TextContent
from _compat.py reached the SDK without being converted to the real
mcp.types.TextContent (same class name, different class object,
pydantic v2 rejects it).

list_tools already converted ToolDef -> mcp.types.Tool at the
protocol boundary. Mirror that pattern for TextContent in call_tool
via _to_mcp_content(). Cold-start cost preserved -- the mcp.types
import stays lazy and server-only.

Test gap closed: prior call_tool integration tests inspected the
returned list directly, never going through SDK validation. New
test builds a real CallToolResult, catching shim leaks on success
path, error path, and meta-tool paths (ts_search, ts_extended).

Reported by @zinkovsky in Mibayy#32.
9-day usage audit (2026-05-17..26, 869 tool calls) surfaced 3 wasteful
patterns. CLAUDE.md and trailing _hints were not enough -- the agent
ignored them. Move the corrections to where they're hard to miss.

1) Chain nudges (call_tool)
   42 find_symbol(X) -> get_function_source(X) and 26 find_symbol(X) ->
   get_full_context(X) chains on the same symbol within 60s, all foldable
   into one get_full_context call. Detector: rolling 8-call buffer per
   server, 60s window. On match, prepend a [NUDGE] TextContent block at
   the top of the response so it survives downstream compression.
   Opt-out: TOKEN_SAVIOR_CHAIN_NUDGE=0.

2) ts_search warm-up (warm_up_async)
   ts_search averaged 4867ms over 19 calls; the Nomic load + 66 tool
   description embeddings dominate the first call. Spawn a background
   thread at server startup so the first client call hits a populated
   cache. Cold load stays silent on failure (substring fallback exists).
   Opt-out: TOKEN_SAVIOR_NO_WARMUP=1.

3) set_project_root nudge
   The cheap path already exists (32% of calls hit it). Tighten the
   response so the agent learns to call switch_project next time --
   that's the documented entry point in CLAUDE.md and one round-trip
   lighter than set_project_root.

Tests: 1764 passed (12 new). test_chain_nudge fixture cleans up tmp_path
slots from _slot_mgr to avoid polluting memory_viewer's active-project
resolution.
…/9d)

9-day data showed 187 occurrences of get_function_source(X) or
get_class_source(X) followed by get_full_context(X) within 60s. The
first read is wasted -- get_full_context re-fetches the source as part
of its bundle. Extends the detector built in v4.4.0 (42 cases of
find_symbol -> get_function_source) with a second pattern, multiplying
the practical coverage by ~5x.

Also snapshot _tool_call_counts in the chain-nudge test fixture so the
suite doesn't push the global past the 15-call navigation-overuse
threshold and flip code_nav._stop_hint behavior in test_query_api.
- persist registered projects + cache-aware set/switch_project (kills
  set_project_root churn: 51 calls, 14.6s outlier, collector-crypt-scanner 20x)
- nudge get_edit_context before edits (0/199 adoption)
- nudge ts_execute on 5+ nav calls in a window (41 calls despite thousands unitary)
- persist ts_search tool-desc embeddings (cold start 5.7s -> skips re-embed)
- silence no-observations hook spam (3578 lines in hook-errors.log)

1771 tests pass (+6 new).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- daemon_client.py: minimal Unix-socket client (best-effort, None on failure)
- _handle_ts_search delegates the first cold call to a warm ts _daemon-serve
  (~130ms) instead of the ~5.7s in-process Nomic load; opt-in via
  TS_SEARCH_COLD_DELEGATE=1, falls back to in-process on any daemon failure
- cli._daemon_serve routes ts_search through its handler (was 'unknown tool')

1779 tests pass (+8 new).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…4.0)

The hardcoded string sat at 3.4.0 through the entire v4.x line, which masked
that the daily-driver venv was running a stale build. Now sourced from
importlib.metadata so it tracks pyproject.toml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mon delegation

- scripts/ts_audit.py: automates the usage audit (latency, chains, adoption
  gaps, nudge fires, ML liveness); baseline snapshot for measuring nudge effect
- telemetry.record_nudge/nudge_counts: persist per-kind nudge fires so
  effectiveness can be tracked vs adoption over successive audits
- TCA revival: flush_session was never called (session_count stuck at 0 all
  deployment) -> now flushed at switch_project + atexit. Broken feature, not
  dead code; made live instead of removed
- ts-daemon.service: launch venv python (has fastembed) so delegated ts_search
  is embedding-quality warm (~23ms) instead of substring

1786 tests pass (+7 new).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- server_handlers/resources.py: list + read observations as MCP resources
  (bounded, ranked by memory_index score)
- wired in main() via list_resources/read_resource, opt-out TS_RESOURCES_DISABLED=1
- read-only, additive; tool path untouched

1792 tests pass (+6 new).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit over 425 real sessions: get_edit_context 0/219 edits, its nudge fired
12x and converted 0. Stop nudging; fold the value into the edit result.
_edit_impact_notice() appends callers + impacted tests to every successful
symbol edit (reuses get_dependents + find_impacted_test_files). Pattern-3 nudge
retired. Opt out with TOKEN_SAVIOR_EDIT_IMPACT=0. 1790 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
find_symbol only ever searched functions and classes, but a miss returned
`complete: True` and its docstring told callers they "do not need to fall
back to grep/search_codebase". For anything that is not a def or a class —
a module global, a constant, a class attribute — that is a confidently
wrong answer. `pfb`, a module-level dict referenced 317 times in one file,
reported as "symbol not found, scanned_files: 2085, complete: true".

Two changes:

1. A miss now reports `searched` (the kinds actually scanned) and, when an
   indexed kind was left out, `not_searched` plus a ready-to-run
   `retry_with` call. `complete: True` stays on the hit path, where it is
   true. The MCP `_suggestion` now names the retry before sending the
   caller to grep.

2. `find_symbol(name, kinds=[...])` accepts `function`, `class`, and
   `variable`; variables cover module globals, constants (UPPER_SNAKE or
   `Final[...]`), and class attributes. The Python annotator extracts them
   from `Assign`/`AnnAssign` at module and class scope — function locals
   are skipped, since they are not addressable from another file.
   `AugAssign` is skipped too: it rebinds rather than defines.

Bindings live in a separate `ProjectIndex.variable_table` (name and
qualified name -> defining files) rather than `symbol_table`, so a global
can never shadow a same-named function in the resolvers that walk the
symbol table, and same-named globals across files report as ambiguous
instead of silently picking one.

`TOKEN_SAVIOR_VARIABLES` gates both halves: `off` skips extraction and
rejects an explicit `kinds=["variable"]` (rather than returning an empty
result that reads as "no such variable"), `index` (default) extracts but
searches only on request, `search` also puts variables in the default
kinds.

The new `variables` field is persisted, so `_CACHE_VERSION` goes 2 -> 3;
a stale v2 cache would otherwise report every project as variable-free.

Other annotators leave `variables` empty for now, which `searched` /
`not_searched` reports honestly rather than hiding.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
andrebrait added a commit to pfBlockerNG/pfBlockerNG that referenced this pull request Jul 22, 2026
Picks up two fork commits. `find_symbol` now reports which symbol kinds it
actually searched instead of claiming an exhaustive scan on a miss, and it
can search module-level and class-level variables via
`find_symbol(name, kinds=["variable"])` — previously only functions and
classes were ever indexed, so a global like `$pfb` / `pfb` returned
"not found, complete: true".

Python, PHP, and shell annotators all extract variables. Indexed by
default; searched on request. `TOKEN_SAVIOR_VARIABLES=search` puts them in
the default kinds, `off` disables both halves.

Indexing this repo yields 4255 variables across 861 files. The first
session after this lands rebuilds the shared venv, as any TS_SOURCE change
does.

Upstream: Mibayy/token-savior#68 (Python half; the PHP and shell halves
depend on the not-yet-merged annotator PRs #50/#51).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Mibayy

Mibayy commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Merged as 30daf13 on main, with you as the commit author.

Not via the merge button, and the reason matters: main had its history rewritten today to purge a leaked credential and some customer data from the public repo. Your branch predates that rewrite, so merging it would have pushed the purged commits straight back in. I cherry-picked your commits onto the new main instead. Same content, same authorship, no reintroduction.

Full suite green (2044 passed) and CI green on main.

If you send more PRs, please rebase on the current main first, otherwise the diff will look enormous and the merge button stays unsafe.

Thanks for the work, it was accurate and well tested.

@Mibayy Mibayy closed this Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants