Why is there no `breakpoint` MCP tool?

[ls-1801]

I am curious why the uexperimental mcp serve does not expose a breakpoint tool?

That is a very severe limitation to me, making the tool pretty useless. My application creates a result that is checked against an expected result at a later point in time on a different thread, without any shared memory that could be tracked with last (i.e., the result is written to a file and loaded later).
Every time I attempted to use Claude or Gemini, they gave up after telling me that UDB can not do forward debugging. After checking the instructions.txt and looking at the exposed tools, why is that not allowed?

[barisione]

When LLMs are exposed to the full set of UDB commands, they get confused and start doing silly things. In particular:

• They set breakpoints, forget about them, then get confused when they hit them. 🤦‍♂️
• If forward operations are exposed, LLMs seem to only use those. We suspect this is because most debuggers aren't time-travel debuggers, so they prefer the more familiar methods in their training set. (We see this with human users as well!)

Do you have more details on what Claude and Gemini say? Is your code open source?

There are definitely more tools that could be exposed to LLMs, but we need to be careful not to actually make things worse by exposing more.

[ls-1801]

@barisione, I am a little hesitant to share my Claude history because it's pretty large, and I cannot really confirm that it does not contain any sensitive information. I'll attach a shortened (still long) section of the chat. I used an LLM to transform the JSON into a more readable format. My instructions were vague because, within the same session, I fixed a crash using the same method (using udb mcp), which worked flawlessly. However, the cause of the crash was relatively easy to identify, based on the backtrace and a single last command.

Yes my code is open source https://github.com/nebulastream/nebulastream/
I was on the recent main branch: 5bf2f730f9b141f406e86a378c78ec8165df533b

I used live-record to find a test execution where the test failed.

 live-record \
            --record-on "symbol:_ZN3NES7Systest23runQueriesAtLocalWorkerERKNSt3__16vectorINS0_12SystestQueryENS1_9allocatorIS3_EEEEmRKNS_29SingleNodeWorkerConfigurationE" \
            --retry-for 4h \
            --thread-fuzzing \
            --disable-aslr \
            --save-on error \
            /home/user/nebulastream/cmake-build-debug/nes-systests/systest/systest \
            "-n" "1" \
            "-t" "JoinEquivalence.test:1" \
            "--workingDir=$WORKING_DIR" \
            "--data" "/home/user/nebulastream/cmake-build-debug/nes-systests/testdata" \
            "--" \
            "--worker.query_engine.number_of_worker_threads=2" \
            "--worker.default_query_execution.execution_mode=INTERPRETER" \
            "--worker.number_of_buffers_in_global_buffer_manager=20000" \
            "--worker.default_query_execution.join_strategy=HASH_JOIN" \
            > "$LOG_FILE" 2>&1

conversation_formatted.md

[mark-undoio]

@ls-1801 Hi - it's very useful to see the chat session, thanks.

The LLM hasn't done a great job of explaining what it can and can't do (and why) - that's something we can work on. It's correct that this tooling doesn't give it access to certain functionality but it obviously shouldn't extend that to claims about UDB's core functionality.

Actually, though, I think the place things went off track was quite early.

Tracing your session

Early on in your session the tool jumped to the end of recorded history to find out how the program exited, retrieving this backtrace:

#0  __GI__exit (status=6006) at ../sysdeps/unix/sysv/linux/_exit.c:30
#1  0x00007ffff7b72216 in __run_exit_handlers (status=6006, listp=0x7ffff7d17680 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at ./stdlib/exit.c:141
#2  0x00007ffff7b7235a in __GI_exit (status=<optimized out>) at ./stdlib/exit.c:148
#3  0x00007ffff7b59caf in __libc_start_call_main (main=main@entry=0x55555933d8b0 <main(int, char const**)>, argc=argc@entry=15, argv=argv@entry=0x7fffffffdd68) at ../sysdeps/nptl/libc_start_call_main.h:74
#4  0x00007ffff7b59d65 in __libc_start_main_impl (main=0x55555933d8b0 <main(int, char const**)>, argc=15, argv=0x7fffffffdd68, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd58) at ../csu/libc-start.c:360
#5  0x000055555933d7e1 in _start ()\n

It then worked back within that backtrace until it got to __libc_start_main_impl (it printed this hypothesis as its reason for doing so): Stepping back to libc_start_call_main to find what the main function returned.

This is what was returned to the LLM when it landed there.

0x00007ffff7b59caa in __libc_start_call_main (main=main@entry=0x55555933d8b0 <main(int, char const**)>, argc=argc@entry=15, argv=argv@entry=0x7fffffffdd68) at ../sysdeps/nptl/libc_start_call_main.h:74
warning: 74    ../sysdeps/nptl/libc_start_call_main.h: No such file or directory

Function: __libc_start_call_main

Source context unavailable.

The "Source context unavailable" is likely a problem we need to solve first, since all the interaction between the MCP server and the agent follows on the from context of it not getting what it wanted at this point.

The agent wants to step back into your main function but without the source for your C runtime (which would likely come via a debug / dev package on your distro, or use of debuginfod) it won't be able to figure out how to do it correctly.

As a result, I think it thrashed around a lot and then tried to answer the question by reading your source code instead.

Why did the previous session work?

In contrast, your previous successful use of the tool involved a crash so I think you'd have sidestepped this issue - it wouldn't have needed to figure out C runtime internals to get back to your code from a segfault / assertion failure / exception.

Workaround

The way I usually prevent this sort of issue is either:

1. Enable debug information / source code for system libraries - on Fedora I just enable debuginfod and it gets me both but different distros vary in their integration for this. If you tell us what distro you're on we may be able to advise you.
2. Manually get the program back to the end of the main() function and then set time-limits max to set a new end-of-time - this avoids the LLM needing to step back into main on its own. You don't really lose anything by doing this if your program exited tidily (even with an error code).

By the way, you can always set the min / max time limits to constrain the agent to only investigate a particular sub-range of a recording. This is just a particularly useful application of it.

Next steps

I'd be interested to see if applying one of the above approaches gets you further.

At this point we're still adding tools gradually to the integration, once we understand what the LLMs really need, so your feedback is very much appreciated! As @barisione said, if we just give them access to all the functionality at our disposal they tend to get confused and become less clever at debugging, so it's a delicate balance.