Bug Fixes & Improvements: Windows Stdio hangs, Cascading Gitignore, UTF-8 Snippets

[tarun1790]

codebase-memory-mcp Codebase Analysis & Improvements Report

This report outlines 5 high-impact bugs, build failures, and architectural issues identified in the DeusData/codebase-memory-mcp repository, along with their root causes and suggested code fixes.

---

1. Windows stdio MCP Handshake Hangs & Early Exits

Important

Impact: Prevents the MCP server from running in stdio mode on Windows when spawned as a subprocess by clients like Cursor or Claude Code.

Root Cause

In mcp.c, the event loop polls stdin for new data to manage idle timeouts. On Windows, it tries to wait directly on the stdin file descriptor handle:

#ifdef _WIN32
        /* Windows: WaitForSingleObject on stdin handle */
        HANDLE hStdin = (HANDLE)_get_osfhandle(fd);
        DWORD wr = WaitForSingleObject(hStdin, STORE_IDLE_TIMEOUT_S * MCP_TIMEOUT_MS);
        if (wr == WAIT_FAILED) {
            break;
        }

Windows anonymous pipes (which redirect standard inputs for subprocesses) do not support synchronization and cannot be used with wait functions. Thus, WaitForSingleObject fails immediately and returns WAIT_FAILED, causing the server loop to break and terminate the process during handshake.

Additionally, stdout is block-buffered on Windows when redirected to a pipe. Although fflush is called in the loop, if the process crashes or hangs on a blocked read, standard outputs remain stuck in the OS-level stream buffer.

Suggested Fix

1. Disable WaitForSingleObject on pipe handles or use PeekNamedPipe to non-blockingly check if input is available:

   #ifdef _WIN32
           HANDLE hStdin = (HANDLE)_get_osfhandle(fd);
           DWORD type = GetFileType(hStdin);
           if (type == FILE_TYPE_PIPE) {
               // Bypass WaitForSingleObject for anonymous pipes to prevent WAIT_FAILED
               DWORD bytes_avail = 0;
               if (PeekNamedPipe(hStdin, NULL, 0, NULL, &bytes_avail, NULL)) {
                   if (bytes_avail == 0) {
                       cbm_usleep(10000); // Back off 10ms to prevent CPU pinning
                       continue;
                   }
               }
           } else {
               DWORD wr = WaitForSingleObject(hStdin, STORE_IDLE_TIMEOUT_S * MCP_TIMEOUT_MS);
               if (wr == WAIT_FAILED) break;
               if (wr == WAIT_TIMEOUT) {
                   cbm_mcp_server_evict_idle(srv, STORE_IDLE_TIMEOUT_S);
                   continue;
               }
           }
   #endif

2. Disable buffering on stdout at server startup in main.c:

   setvbuf(stdout, NULL, _IONBF, 0);

---

2. Cascading .gitignore Files Ignored in Subdirectories

Warning

Impact: Subdirectory indexing ignores .gitignore files nested inside subfolders, causing build/dist folders or other excluded patterns to be scanned.

Root Cause

In discover.c, try_load_nested_gitignore checks:

static cbm_gitignore_t *try_load_nested_gitignore(const walk_frame_t *frame) {
    if (frame->local_gi || frame->prefix[0] == '\0') {
        return NULL;
    }
    ...
}

If a parent directory has a .gitignore loaded, frame->local_gi is set. When entering subdirectories, try_load_nested_gitignore exits early, meaning nested .gitignore files deeper in the tree are never loaded.

Suggested Fix

Allow loading nested .gitignore files even if a parent .gitignore was loaded. If a nested .gitignore is found in the current directory, merge its rules with the parent or stack them in a chain of active ignore filters.

---

3. get_code_snippet Hangs on Non-UTF-8 Encoded Files

Important

Impact: MCP clients (e.g. Node-based clients) crash or hang when requesting snippets from files using non-UTF-8 encodings (such as EUC-KR / CP949 or Shift-JIS).

Root Cause

The read_file_lines function reads raw bytes from files. When a non-UTF-8 file is read, it inserts invalid UTF-8 bytes directly into the JSON response structure.
yy_doc_to_str serializes the document using YYJSON_WRITE_ALLOW_INVALID_UNICODE which allows invalid bytes to be written straight to standard output. When the receiving MCP client (which expects strictly valid UTF-8 JSON) parses the stdout stream, it chokes on the invalid bytes and hangs.

Suggested Fix

Sanitize the file source content in build_snippet_response before adding it to the JSON response. Validate the UTF-8 sequences and replace any invalid bytes with the Unicode replacement character (\uFFFD or ?) to guarantee the JSON string output is always valid UTF-8.

---

4. libgit2 1.8+ Compilation Failure

Tip

Impact: Compilation fails when building with newer libgit2 versions (v1.8.0+).

Root Cause

The git_allocator struct was moved from <git2.h> to <git2/sys/alloc.h> in libgit2 1.8+. Because cbm.c only includes <git2.h>, compiling with libgit2 1.8+ throws an incomplete type error for git_allocator.

Suggested Fix

Include <git2/sys/alloc.h> conditionally in cbm.c:

 #if defined(HAVE_LIBGIT2)
 #include <git2.h>
+#include <git2/sys/alloc.h> // For git_allocator on libgit2 1.8+
 #endif

---

5. Directory Discovery Ignores .git/info/exclude

Note

Impact: Local-only ignore rules configured in .git/info/exclude are not respected, leading to unwanted local files being indexed.

Root Cause

The discovery process in src/discover/discover.c searches for .gitignore and .cbmignore files but never attempts to load .git/info/exclude.

Suggested Fix

During setup, check for the presence of .git/info/exclude in the repository root and load it as a base gitignore filter alongside .gitignore.

[tarun1790]

codebase-memory-mcp Codebase Analysis & Improvements Report (Batch 2)

This comment outlines two additional bugs/improvements identified in the codebase, along with their root causes and suggested code-level fixes (complete with code diffs).

---

1. ADR Data Loss on Index Rebuilds / Incremental runs

Important

Impact: Users silently lose all their stored Architecture Decision Records (ADRs) whenever a full re-indexing or an incremental indexing pass runs.

Root Cause

Both full re-indexing (in src/pipeline/pipeline.c) and incremental indexing (in src/pipeline/pipeline_incremental.c) delete the existing SQLite database using cbm_unlink(db_path) and then write the new database from scratch.
Because the custom page-based sqlite_writer only serializes the graph buffer nodes/edges/hashes, the project_summaries table (which stores ADRs) is recreated empty. Consequently, the user's stored ADRs are wiped out.

Suggested Fix

Temporarily back up the ADR metadata (the content, created_at, and updated_at timestamps) in memory before the SQLite file is unlinked, and restore it using a new cbm_store_adr_restore function right after the new SQLite database is generated. This preserves both the ADR content and the original creation/update timestamps.

Code Diff

diff --git a/src/store/store.h b/src/store/store.h
index 26b09a5..98749b3 100644
--- a/src/store/store.h
+++ b/src/store/store.h
@@ -544,6 +544,7 @@ typedef struct {
 } cbm_adr_t;
 
 int cbm_store_adr_store(cbm_store_t *s, const char *project, const char *content);
+int cbm_store_adr_restore(cbm_store_t *s, const char *project, const char *content, const char *created_at, const char *updated_at);
 int cbm_store_adr_get(cbm_store_t *s, const char *project, cbm_adr_t *out);
 int cbm_store_adr_delete(cbm_store_t *s, const char *project);
 int cbm_store_adr_update_sections(cbm_store_t *s, const char *project, const char **keys,
diff --git a/src/store/store.c b/src/store/store.c
index c237332..f0b7693 100644
--- a/src/store/store.c
+++ b/src/store/store.c
@@ -5359,6 +5359,26 @@ int cbm_store_adr_store(cbm_store_t *s, const char *project, const char *content
     return (rc == SQLITE_DONE) ? CBM_STORE_OK : CBM_STORE_ERR;
 }
 
+int cbm_store_adr_restore(cbm_store_t *s, const char *project, const char *content, const char *created_at, const char *updated_at) {
+    const char *sql =
+        "INSERT INTO project_summaries (project, summary, source_hash, created_at, updated_at) "
+        "VALUES (?1, ?2, '', ?3, ?4) "
+        "ON CONFLICT(project) DO UPDATE SET summary=excluded.summary, "
+        "created_at=excluded.created_at, updated_at=excluded.updated_at";
+    sqlite3_stmt *stmt = NULL;
+    if (sqlite3_prepare_v2(s->db, sql, CBM_NOT_FOUND, &stmt, NULL) != SQLITE_OK) {
+        store_set_error_sqlite(s, "adr_restore");
+        return CBM_STORE_ERR;
+    }
+    bind_text(stmt, 1, project);
+    bind_text(stmt, 2, content);
+    bind_text(stmt, 3, created_at ? created_at : "");
+    bind_text(stmt, 4, updated_at ? updated_at : "");
+    int rc = sqlite3_step(stmt);
+    sqlite3_finalize(stmt);
+    return (rc == SQLITE_DONE) ? CBM_STORE_OK : CBM_STORE_ERR;
+}
+
 int cbm_store_adr_get(cbm_store_t *s, const char *project, cbm_adr_t *out) {

---

2. Cross-Service Edge Traversal Ignored in trace_path

Warning

Impact: The trace_path tool with mode=cross_service fails to navigate across repositories because it completely ignores all CROSS_* edges.

Root Cause

In src/mcp/mcp.c, the mode_cross_svc array is hardcoded to:

static const char *mode_cross_svc[] = {"HTTP_CALLS", "ASYNC_CALLS", "DATA_FLOWS", "CALLS"};

This ignores the CROSS_ prefix and does not cover other cross-service protocols (like CROSS_GRPC_CALLS, CROSS_GRAPHQL_CALLS, CROSS_TRPC_CALLS, CROSS_CHANNEL).

Suggested Fix

Update the mode_cross_svc array to include all actual CROSS_* edge type names generated by the pipeline, and adjust the defaults count from 4 to 10.

Code Diff

diff --git a/src/mcp/mcp.c b/src/mcp/mcp.c
index 8102b1e..43b6ebf 100644
--- a/src/mcp/mcp.c
+++ b/src/mcp/mcp.c
@@ -30,6 +30,7 @@ enum {
     MCP_BFS_LIMIT = 100,
     MCP_N_DEFAULTS_2 = 2,
     MCP_N_DEFAULTS_4 = 4,
+    MCP_N_DEFAULTS_10 = 10,
     MCP_URI_PREFIX = 7,      /* strlen("file://") */
     MCP_CONTENT_PREFIX = 15, /* strlen("Content-Length:") */
     MCP_RETURN_2 = 2,
@@ -2164,7 +2165,11 @@ static yyjson_doc *resolve_trace_edge_types(const char *args, const char *mode,
                                             const char **out_types, int *out_count) {
     static const char *mode_calls[] = {"CALLS"};
     static const char *mode_data_flow[] = {"CALLS", "DATA_FLOWS"};
-    static const char *mode_cross_svc[] = {"HTTP_CALLS", "ASYNC_CALLS", "DATA_FLOWS", "CALLS"};
+    static const char *mode_cross_svc[] = {
+        "CROSS_HTTP_CALLS", "CROSS_ASYNC_CALLS", "CROSS_CHANNEL",
+        "CROSS_GRPC_CALLS", "CROSS_GRAPHQL_CALLS", "CROSS_TRPC_CALLS",
+        "HTTP_CALLS", "ASYNC_CALLS", "DATA_FLOWS", "CALLS"
+    };
 
     *out_count = 0;
 
@@ -2196,7 +2201,7 @@ static yyjson_doc *resolve_trace_edge_types(const char *args, const char *mode,
         n_defaults = MCP_N_DEFAULTS_2;
     } else if (mode && strcmp(mode, "cross_service") == 0) {
         defaults = mode_cross_svc;
-        n_defaults = MCP_N_DEFAULTS_4;
+        n_defaults = MCP_N_DEFAULTS_10;
     }

[DeusData]

Hey thanks for highlighting. Will check everything beginning of next week.

[tarun1790]

codebase-memory-mcp Codebase Analysis & Improvements Report (Batch 3)

This comment outlines the root cause and complete code-level fix (including a clean code diff) for the cascading gitignore bug.

---

1. Cascading .gitignore Files Ignored in Subdirectories

Warning

Impact: Nested .gitignore files inside subfolders (e.g. sub/nested/.gitignore) are completely ignored if any parent directory already has a .gitignore loaded. This causes build/dist folders or other excluded patterns nested inside subfolders to be scanned during indexing.

Root Cause

In src/discover/discover.c, when walking the directory structure, the recursive framework uses a stack of frames (walk_frame_t).
Each frame has a local_gi pointer and a local_gi_prefix string.
When entering a subdirectory, the parent's local_gi and local_gi_prefix are inherited by the child frame.
However, the try_load_nested_gitignore helper checks:

static cbm_gitignore_t *try_load_nested_gitignore(const walk_frame_t *frame) {
    if (frame->local_gi || frame->prefix[0] == '\0') {
        return NULL;
    }

If frame->local_gi is already non-NULL (because a parent .gitignore was loaded and inherited), the function returns early and does not look for nested .gitignore files in the current folder.

Suggested Fix

1. Define a list node structure cbm_local_gi_node_t to chain active gitignores and their relative prefixes.
2. Update walk_frame_t to point to the head of this chain (local_gi_chain).
3. Inherit this chain during the directory walk by assigning stack[*top].local_gi_chain = parent->local_gi_chain.
4. In try_load_nested_gitignore, check for a .gitignore in the current frame directory regardless of whether parent gitignores are already active.
5. Prepend any loaded nested gitignore to the current frame's chain.
6. Cleanly track and free all allocated nodes at the end of walk_dir to prevent leaks.
7. Iterate and match against all gitignores in the chain in should_skip_directory and should_skip_file.

Code Diff

diff --git a/src/discover/discover.c b/src/discover/discover.c
index 314c00c..f7355df 100644
--- a/src/discover/discover.c
+++ b/src/discover/discover.c
@@ -270,20 +270,25 @@ static const char *local_rel_path(const char *rel_path, const char *local_prefix
     return rel_path;
 }
 
+typedef struct cbm_local_gi_node {
+    const cbm_gitignore_t *gi;
+    char prefix[CBM_SZ_4K];
+    struct cbm_local_gi_node *next;
+} cbm_local_gi_node_t;
+
 /* Check if a directory entry should be skipped (hardcoded dirs + gitignore). */
 static bool should_skip_directory(const char *entry_name, const char *rel_path,
                                   const cbm_discover_opts_t *opts, const cbm_gitignore_t *gitignore,
-                                  const cbm_gitignore_t *cbmignore, const cbm_gitignore_t *local_gi,
-                                  const char *local_gi_prefix) {
+                                  const cbm_gitignore_t *cbmignore, const cbm_local_gi_node_t *local_gi_chain) {
     if (cbm_should_skip_dir(entry_name, opts ? opts->mode : CBM_MODE_FULL)) {
         return true;
     }
     if (gitignore && cbm_gitignore_matches(gitignore, rel_path, true)) {
         return true;
     }
-    if (local_gi) {
-        const char *lrel = local_rel_path(rel_path, local_gi_prefix);
-        if (cbm_gitignore_matches(local_gi, lrel, true)) {
+    for (const cbm_local_gi_node_t *curr = local_gi_chain; curr != NULL; curr = curr->next) {
+        const char *lrel = local_rel_path(rel_path, curr->prefix);
+        if (cbm_gitignore_matches(curr->gi, lrel, true)) {
             return true;
         }
     }
@@ -296,8 +301,8 @@ static bool should_skip_directory(const char *entry_name, const char *rel_path,
 /* Check if a regular file should be skipped (filters + gitignore + size). */
 static bool should_skip_file(const char *entry_name, const char *rel_path,
                              const cbm_discover_opts_t *opts, const cbm_gitignore_t *gitignore,
-                             const cbm_gitignore_t *cbmignore, const cbm_gitignore_t *local_gi,
-                             const char *local_gi_prefix, off_t file_size) {
+                             const cbm_gitignore_t *cbmignore, const cbm_local_gi_node_t *local_gi_chain,
+                             off_t file_size) {
     cbm_index_mode_t mode = opts ? opts->mode : CBM_MODE_FULL;
     if (cbm_has_ignored_suffix(entry_name, mode)) {
         return true;
@@ -311,9 +316,9 @@ static bool should_skip_file(const char *entry_name, const char *rel_path,
     if (gitignore && cbm_gitignore_matches(gitignore, rel_path, false)) {
         return true;
     }
-    if (local_gi) {
-        const char *lrel = local_rel_path(rel_path, local_gi_prefix);
-        if (cbm_gitignore_matches(local_gi, lrel, false)) {
+    for (const cbm_local_gi_node_t *curr = local_gi_chain; curr != NULL; curr = curr->next) {
+        const char *lrel = local_rel_path(rel_path, curr->prefix);
+        if (cbm_gitignore_matches(curr->gi, lrel, false)) {
             return true;
         }
     }
@@ -384,9 +389,9 @@ static int safe_stat(const char *abs_path, struct stat *st) {
 /* Process a single regular file entry during directory walk. */
 static void walk_dir_process_file(const char *abs_path, const char *rel_path, const char *name,
                                   const cbm_discover_opts_t *opts, const cbm_gitignore_t *gitignore,
-                                  const cbm_gitignore_t *cbmignore, const cbm_gitignore_t *local_gi,
-                                  const char *local_gi_prefix, off_t size, file_list_t *out) {
-    if (should_skip_file(name, rel_path, opts, gitignore, cbmignore, local_gi, local_gi_prefix,
+                                  const cbm_gitignore_t *cbmignore, const cbm_local_gi_node_t *local_gi_chain,
+                                  off_t size, file_list_t *out) {
+    if (should_skip_file(name, rel_path, opts, gitignore, cbmignore, local_gi_chain,
                          size)) {
         return;
     }
@@ -400,14 +405,13 @@ static void walk_dir_process_file(const char *abs_path, const char *rel_path, co
 typedef struct {
     char dir[CBM_SZ_4K];
     char prefix[CBM_SZ_4K];
-    cbm_gitignore_t *local_gi;       /* nested .gitignore for this subtree */
-    char local_gi_prefix[CBM_SZ_4K]; /* rel_prefix when local_gi was loaded */
+    const cbm_local_gi_node_t *local_gi_chain;
 } walk_frame_t;
 #define WALK_STACK_CAP 512
 /* Build abs/rel paths and process one directory entry. */
 /* Try to load a nested .gitignore from this directory. Returns owned pointer or NULL. */
 static cbm_gitignore_t *try_load_nested_gitignore(const walk_frame_t *frame) {
-    if (frame->local_gi || frame->prefix[0] == '\0') {
+    if (frame->prefix[0] == '\0') {
         return NULL;
     }
     char gi_path[CBM_SZ_4K];
@@ -427,8 +431,7 @@ static void walk_push_subdir(walk_frame_t *stack, int *top, const char *abs_path
     }
     snprintf(stack[*top].dir, CBM_SZ_4K, "%s", abs_path);
     snprintf(stack[*top].prefix, CBM_SZ_4K, "%s", rel_path);
-    stack[*top].local_gi = parent->local_gi;
-    snprintf(stack[*top].local_gi_prefix, CBM_SZ_4K, "%s", parent->local_gi_prefix);
+    stack[*top].local_gi_chain = parent->local_gi_chain;
     (*top)++;
 }
 
@@ -453,7 +456,7 @@ static void walk_dir_process_entry(cbm_dirent_t *entry, const walk_frame_t *fram
 
     if (S_ISDIR(st.st_mode)) {
         if (!should_skip_directory(entry->name, rel_path, opts, gitignore, cbmignore,
-                                   frame->local_gi, frame->local_gi_prefix)) {
+                                   frame->local_gi_chain)) {
             walk_push_subdir(stack, top, abs_path, rel_path, frame);
         } else {
             /* Record the excluded subtree root so callers can report it (#411). */
@@ -461,7 +464,7 @@ static void walk_dir_process_entry(cbm_dirent_t *entry, const walk_frame_t *fram
         }
     } else if (S_ISREG(st.st_mode)) {
         walk_dir_process_file(abs_path, rel_path, entry->name, opts, gitignore, cbmignore,
-                              frame->local_gi, frame->local_gi_prefix, st.st_size, out);
+                              frame->local_gi_chain, st.st_size, out);
     }
 }
 
@@ -479,9 +482,13 @@ static void walk_dir(const char *dir_path, const char *rel_prefix, const cbm_dis
     cbm_gitignore_t *owned_gis[GI_OWNED_CAP];
     int owned_count = 0;
 
+    cbm_local_gi_node_t *owned_nodes[WALK_STACK_CAP];
+    int owned_nodes_count = 0;
+
     int top = 0;
     snprintf(stack[top].dir, CBM_SZ_4K, "%s", dir_path);
     snprintf(stack[top].prefix, CBM_SZ_4K, "%s", rel_prefix);
+    stack[top].local_gi_chain = NULL;
     top++;
 
     while (top > 0) {
@@ -489,10 +496,22 @@ static void walk_dir(const char *dir_path, const char *rel_prefix, const cbm_dis
 
         cbm_gitignore_t *loaded = try_load_nested_gitignore(&frame);
         if (loaded) {
-            frame.local_gi = loaded;
-            snprintf(frame.local_gi_prefix, sizeof(frame.local_gi_prefix), "%s", frame.prefix);
+            cbm_local_gi_node_t *node = malloc(sizeof(cbm_local_gi_node_t));
+            if (node) {
+                node->gi = loaded;
+                snprintf(node->prefix, sizeof(node->prefix), "%s", frame.prefix);
+                node->next = frame.local_gi_chain;
+                frame.local_gi_chain = node;
+                if (owned_nodes_count < WALK_STACK_CAP) {
+                    owned_nodes[owned_nodes_count++] = node;
+                } else {
+                    free(node);
+                }
+            }
             if (owned_count < GI_OWNED_CAP) {
                 owned_gis[owned_count++] = loaded;
+            } else {
+                cbm_gitignore_free(loaded);
             }
         }
 
@@ -510,6 +529,9 @@ static void walk_dir(const char *dir_path, const char *rel_prefix, const cbm_dis
     for (int i = 0; i < owned_count; i++) {
         cbm_gitignore_free(owned_gis[i]);
     }
+    for (int i = 0; i < owned_nodes_count; i++) {
+        free(owned_nodes[i]);
+    }
     free(stack);
 }

[DeusData]

Hey, thanks for the great analysis. Will check everything ASAP 😊

[DeusData]

For tracking: this report bundles three separate items that each also have focused issues / PRs —

• Windows stdio handshake hang overlaps Windows stdio MCP handshake hangs — stdout only flushes on stdin EOF (v0.8.1) #513 (same WaitForSingleObject-on-anonymous-pipe root cause you describe).
• Cascading / non-root .gitignore overlaps .gitignore (non repo root) gaps and overrides #510 (PR fix(discover): honor root .gitignore when indexing subfolders #517 honors the root .gitignore when indexing subfolders).
• UTF-8 / JSON snippet escaping is addressed by PR fix(foundation): properly escape JSON control characters as \u00XX #527 (escape JSON control characters as \u00XX).

Labeling bug / windows. Going forward, atomic one-bug-per-issue reports are easier to review, bisect, and revert — but thank you for the thorough analysis here; it's genuinely useful.