diff --git a/.claude/skills/compile/SKILL.md b/.claude/skills/compile/SKILL.md new file mode 100644 index 000000000..7ae797fa1 --- /dev/null +++ b/.claude/skills/compile/SKILL.md @@ -0,0 +1,140 @@ +--- +name: compile +description: Build ddprof inside the Ubuntu 24 dev Docker container. Defaults to Debug (gcc, no clang-tidy) for speed. Use when the user asks to compile, build, or rebuild the project. Reuses a running container when possible, otherwise launches one in the background. Accepts an optional mode arg (Deb/Rel/DebTidy/San/TSan/Cov). +--- + +# Compile ddprof (Ubuntu 24 dev container) + +Builds **all targets** inside the Ubuntu 24 dev container. Default mode is +**Debug** (gcc, no clang-tidy) — fastest iteration. Override via `args` when +the user wants something else. + +## Args + +`args` is a single optional token: the build mode. + +| `args` | Mode | CC/CXX | CMake helper | Notes | +|------------|-----------|-----------|-------------------|------------------------------------------| +| (empty) | `Deb` | gcc | `DebCMake` | Default. Fastest. No clang-tidy. | +| `rel` | `Rel` | gcc | `RelCMake` | Optimised + LTO. Slow link. | +| `debtidy` | `DebTidy` | clang | `DebTidyCMake` | Enables clang-tidy. Slowest. Lint gate. | +| `san` | `San` | gcc | `SanCMake` | ASan + UBSan. | +| `tsan` | `TSan` | gcc | `TSanCMake` | ThreadSanitizer. | +| `cov` | `Cov` | gcc | `CovCMake` | Coverage instrumentation. | + +Match `args` case-insensitively. If the user's natural-language request +implies a mode (e.g. "do a release build", "run with sanitizers"), prefer +that over the default. Do not set `CC`/`CXX` — `DebTidyCMake` passes the +clang compiler flags to cmake directly; all other modes use the container's +default gcc. + +## Step 1 — Find a running container + +```bash +docker ps \ + --filter ancestor=base_ddprof_24 \ + --format '{{.ID}}' | head -1 +``` + +Capture the output as `CID`. If non-empty → skip to Step 3. + +## Step 2 — Start one in the background (only if nothing is running) + +Confirm an image exists: + +```bash +docker image ls -f reference=base_ddprof_24 -q +``` + +If no image is present, stop and ask the user to run +`./tools/launch_local_build.sh -u 24` once — the first-time image build is +long and interactive. Do not try to build the image automatically. + +If the image exists, start a detached container with the same mounts as the +launch script (minus the interactive TTY and SSH agent): + +```bash +CID=$(docker run -d --rm \ + -u "$(id -u):$(id -g)" \ + --network=host -w /app \ + --cap-add CAP_SYS_PTRACE --cap-add SYS_ADMIN \ + -v "$PWD:/app" \ + base_ddprof_24 \ + sleep infinity) +``` + +Note: uses `sleep infinity` so subsequent `docker exec` calls have a host to +attach to. Let the user reclaim it; do not auto-kill. + +## Step 3 — Run the build via `docker exec` + +Substitute `` and `` from the table above. For modes other than +`DebTidy`, omit the `CC`/`CXX` exports. + +Example for the default (`Deb`): + +```bash +docker exec "$CID" bash -lc ' + set -euo pipefail + cd /app + source ./setup_env.sh + MkBuildDir Deb + DebCMake ../ + ninja +' +``` + +Example for `DebTidy`: + +```bash +docker exec "$CID" bash -lc ' + set -euo pipefail + cd /app + source ./setup_env.sh + MkBuildDir DebTidy + DebTidyCMake ../ + ninja +' +``` + +### Backgrounding & exit codes + +Long builds should run via `Bash` with `run_in_background: true` so the user +isn't blocked. The Step-3 heredoc already runs under `set -euo pipefail`, so +both forms below correctly propagate ninja's exit code: + +- `ninja > /tmp/ddprof_compile.log 2>&1` — simplest, log only +- `ninja 2>&1 | tee /tmp/ddprof_compile.log` — live output + log + +(Outside a `pipefail` shell, `tee` would swallow ninja's failure — redirect +instead in that case.) + +After completion, **check the exit code reported by the task notification +first.** Only inspect the log if it's non-zero. Use the pattern below to find +errors, and fall back to `tail` if nothing matches so a real failure can never look +like success: + +```bash +errs=$(grep -nE "FAILED:|ninja: build stopped|error:|fatal error:|undefined reference|CMake Error|No space left|Killed" \ + /tmp/ddprof_compile.log | head -40) +if [ -n "$errs" ]; then + printf '%s\n' "$errs" +else + # Nothing matched a known marker — dump the tail so the agent never + # silently reports success on a non-zero exit. + tail -60 /tmp/ddprof_compile.log +fi +``` + +Surface the result (path:line + diagnostic) to the user — do not dump the +whole log on success. + +## Notes + +- Build directory is derived from `MkBuildDir ` and the host + libc/compiler triple, e.g. `build_gcc_unknown-linux-2.39_Deb`, + `build_clang_unknown-linux-2.39_DebTidy`. Each mode has its own dir, so + switching modes does not clobber the previous build. +- `ninja` auto-detects core count; no `-j` flag needed. +- For Alpine/release builds, this skill is the wrong tool — see + `CLAUDE.md` § "Alpine (release) builds". diff --git a/AGENTS.md b/AGENTS.md index d4d612177..7d07660e4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,9 +34,6 @@ All development happens inside Docker containers. The source tree is mounted at # Alpine (musl — matches the release binary environment) ./tools/launch_local_build.sh -f ./app/base-env-alpine/Dockerfile -# Clang instead of GCC -./tools/launch_local_build.sh --clang - # Force rebuild the Docker image ./tools/launch_local_build.sh --clean ``` @@ -52,7 +49,7 @@ Day-to-day iteration uses the Ubuntu 24 container. Execute commands inside it: ```bash docker exec -it bash # or run a single command -docker exec bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && make -j$(nproc) ddprof" +docker exec bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && ninja ddprof" ``` ## Build system @@ -90,22 +87,22 @@ source setup_env.sh # Release build (fast, optimised, LTO) MkBuildDir Rel RelCMake ../ -make -j$(nproc) ddprof +ninja ddprof # Debug build (symbols, no optimisation) MkBuildDir Deb DebCMake ../ -make -j$(nproc) +ninja # Sanitized build (ASan + UBSan — catches memory bugs) MkBuildDir San SanCMake ../ -make -j$(nproc) +ninja # Thread sanitizer MkBuildDir TSan TSanCMake ../ -make -j$(nproc) +ninja ``` ### Build modes at a glance @@ -114,8 +111,10 @@ make -j$(nproc) |---|---|---|---| | `Rel` | `RelCMake` | Release | Performance testing, pre-release checks | | `Deb` | `DebCMake` | Debug | Day-to-day debugging, step-through | +| `DebTidy` | `DebTidyCMake` | Debug + clang-tidy | Lint gate (clang only, slowest) | | `San` | `SanCMake` | SanitizedDebug | Catching memory/UB errors | | `TSan` | `TSanCMake` | ThreadSanitizedDebug | Catching data races | +| `Cov` | `CovCMake` | Coverage | Coverage instrumentation | | `AlpRel` | `RelCMake` | Release (Alpine) | **Release binary** — what ships to users | ### Alpine (release) builds @@ -129,7 +128,7 @@ musl binary that runs everywhere. Use the Alpine container: source setup_env.sh MkBuildDir AlpRel RelCMake -DDDPROF_STATIC=ON ../ -make -j$(nproc) ddprof +ninja ddprof ``` The resulting `ddprof` binary is fully static, compatible with both glibc and diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 000000000..47dc3e3d8 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/app/base-env/Dockerfile b/app/base-env/Dockerfile index c40c781be..ad4c2e86e 100644 --- a/app/base-env/Dockerfile +++ b/app/base-env/Dockerfile @@ -1,7 +1,6 @@ # Using a recent compiler version and recent OS (better tooling) # We'll implement libc version sanitization in the code itself ARG UBUNTU_VERSION=22 -ARG COMPILER="gcc" FROM ubuntu:${UBUNTU_VERSION}.04 as base ARG UBUNTU_VERSION @@ -31,11 +30,7 @@ FROM base-${UBUNTU_VERSION} AS base-gcc ENV CC=gcc-${GCC_VERSION} ENV CXX=g++-${GCC_VERSION} -FROM base-${UBUNTU_VERSION} AS base-clang -ENV CC=clang-${CLANG_VERSION} -ENV CXX=clang++-${CLANG_VERSION} - -FROM base-${COMPILER} AS final +FROM base-gcc AS final # Tell docker to use bash as the default SHELL ["/bin/bash", "-c"] diff --git a/setup_env.sh b/setup_env.sh index a487a0abe..a27955b48 100755 --- a/setup_env.sh +++ b/setup_env.sh @@ -57,7 +57,14 @@ GetDefaultAllocatorOptions() { } GetDirectoryExtention() { - echo "_${DDPROF_EXTENSION_CC}_${DDPROF_EXTENSION_OS}_${1}" + # DebTidy is clang-only by design — force the clang segment of the build-dir + # name regardless of CC. The vendor extension is forced separately in + # DebTidyCMake (because CmakeWithOptions calls this with BUILD_TYPE=Debug). + local CC_SUFFIX=${DDPROF_EXTENSION_CC} + if [[ "$1" == "DebTidy" ]]; then + CC_SUFFIX=clang + fi + echo "_${CC_SUFFIX}_${DDPROF_EXTENSION_OS}_${1}" } COMMON_OPT="${COMPILER_SETTING} ${DEFAULT_ALLOCATOR_OPT} -DCMAKE_INSTALL_PREFIX=${DDPROF_INSTALL_PREFIX} -DCOLLATZ_INSTALL_PREFIX=${DDPROF_COLLATZ_INSTALL_PREFIX} -DBUILD_BENCHMARKS=${DDPROF_BUILD_BENCH}" @@ -71,7 +78,7 @@ CmakeWithOptions() { shift local VENDOR_EXTENSION=$(GetDirectoryExtention ${BUILD_TYPE}) # shellcheck disable=SC2086 - cmake_cmd="cmake ${COMMON_OPT} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DVENDOR_EXTENSION=${VENDOR_EXTENSION} $@" + cmake_cmd="cmake -GNinja ${COMMON_OPT} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DVENDOR_EXTENSION=${VENDOR_EXTENSION} $@" echoerr "-------------- cmake command -------------- " echoerr ${cmake_cmd} eval ${cmake_cmd} @@ -87,10 +94,17 @@ DebCMake() { CmakeWithOptions ${BUILD_TYPE} $@ } -# Requires clang as compiler DebTidyCMake() { local BUILD_TYPE=Debug - CmakeWithOptions ${BUILD_TYPE} -DENABLE_CLANG_TIDY=ON $@ + # VENDOR_EXTENSION uses BUILD_TYPE (Debug), so the DebTidy special-case in + # GetDirectoryExtention does not fire here — override locally so vendored + # deps land in _clang_*_Debug and are shared with CC=clang DebCMake builds + # rather than rebuilt for clang-tidy specifically. + local DDPROF_EXTENSION_CC=clang + if [[ ( -n "${CC:-}" && "${CC%-*}" != "clang" ) || ( -n "${CXX:-}" && "${CXX%-*}" != "clang++" ) ]]; then + echoerr "DebTidyCMake: forcing clang/clang++ (ignoring CC=${CC:-} CXX=${CXX:-} — clang-tidy requires clang)." + fi + CmakeWithOptions ${BUILD_TYPE} -DENABLE_CLANG_TIDY=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ $@ } SanCMake() { @@ -109,7 +123,7 @@ CovCMake() { } ## Build a directory with a naming that reflects the OS / compiler we are using -## Example : mkBuildDir Rel --> build_UB18_clang_Rel +## Example: MkBuildDir Rel --> build_gcc_unknown-linux-2.39_Rel MkBuildDir() { local BUILD_DIR_EXTENSION=$(GetDirectoryExtention ${1}) echo ${BUILD_DIR_EXTENSION} diff --git a/test/pthread_deadlock.cc b/test/pthread_deadlock.cc index 5b33cb131..52ed2c7f2 100644 --- a/test/pthread_deadlock.cc +++ b/test/pthread_deadlock.cc @@ -18,21 +18,26 @@ void getattr_loop(bool loop) { } // namespace int main() { + using namespace std::chrono_literals; + constexpr auto kStartupDelay = 10ms; + constexpr auto kRaceWindow = 100ms; + constexpr uint32_t kStopTimeoutMs = 1000; + // Start a thread that calls pthread_getattr_np in a tight loop std::thread t(getattr_loop, true); // Give the thread time to start running - std::this_thread::sleep_for(std::chrono::milliseconds(10)); + std::this_thread::sleep_for(kStartupDelay); // Start profiling while the thread is actively calling pthread_getattr_np - int ret = ddprof_start_profiling(); + int const ret = ddprof_start_profiling(); if (ret != 0) { fprintf(stderr, "Failed to start profiling (ret=%d)\n", ret); return 1; } // Let it run for a bit to exercise the race - std::this_thread::sleep_for(std::chrono::milliseconds(100)); + std::this_thread::sleep_for(kRaceWindow); // Stop the thread g_stop.store(true, std::memory_order_relaxed); @@ -42,9 +47,7 @@ int main() { std::thread t2(getattr_loop, false); t2.join(); - if (ret == 0) { - ddprof_stop_profiling(1000); - } + ddprof_stop_profiling(kStopTimeoutMs); return 0; } diff --git a/test/pthread_deadlock_mmap.cc b/test/pthread_deadlock_mmap.cc index de1304f0a..86e1f619d 100644 --- a/test/pthread_deadlock_mmap.cc +++ b/test/pthread_deadlock_mmap.cc @@ -8,15 +8,18 @@ // (without malloc hook being called because statically linked) // the TLS state is not initialized (otherwise it would deadlock). int main() { + constexpr size_t kLargeAllocBytes = size_t{1024} * 1024 * 16; + constexpr uint32_t kStopTimeoutMs = 1000; + std::latch l(1); std::thread t([&] { l.wait(); // large allocation to exercise the mmap hooks path. - void *p = malloc(1024 * 1024 * 16); + void *p = malloc(kLargeAllocBytes); free(p); }); - int ret = ddprof_start_profiling(); + int const ret = ddprof_start_profiling(); if (ret != 0) { fprintf(stderr, "Failed to start profiling (ret=%d)\n", ret); return 1; @@ -25,9 +28,7 @@ int main() { l.count_down(); t.join(); - if (ret == 0) { - ddprof_stop_profiling(1000); - } + ddprof_stop_profiling(kStopTimeoutMs); return 0; } diff --git a/tools/launch_local_build.sh b/tools/launch_local_build.sh index e0ab0dfc9..a039ad2d8 100755 --- a/tools/launch_local_build.sh +++ b/tools/launch_local_build.sh @@ -32,10 +32,9 @@ usage() { echo " Optional parameters " echo " --dockerfile/-f : use a custom docker file." echo " --clean/-c : rebuild the image before creating it." - echo " --ubuntu_version/-u : specify ubuntu version (expected values: 16 / 18 / 20)" + echo " --ubuntu_version/-u : specify ubuntu version (expected values: 16 / 18 / 20 / 22 / 24)" echo " --image_id/-i : use a specified docker ID, conflicts with -u." - echo " --clang : use clang instead of gcc. - --cap-test : add CAP_SETUID/SETGID/IPC_LOCK/SETFCAP for capability unit tests." + echo " --cap-test : add CAP_SETUID/SETGID/IPC_LOCK/SETFCAP for capability unit tests." } if [ $# != 0 ] && [ "$1" == "-h" ]; then @@ -46,7 +45,6 @@ fi PERFORM_CLEAN=0 # This default is to ensure that users that compile from source are likely to have a compatible libc UBUNTU_VERSION=18 -COMPILER="gcc" EXTRA_CAPS="" USER_OPTION="-u $(id -u):$(id -g)" @@ -75,10 +73,6 @@ while [ $# != 0 ]; do shift shift ;; - --clang) - COMPILER="clang" - shift - ;; --cap-test) # setcap requires CAP_SETFCAP in the effective set, which only root # has by default. Start as root so setcap works; use 'su ' to @@ -124,7 +118,7 @@ fi # If we didn't pass a custom ID, then focus on Ubuntu if [ ! ${CUSTOM_ID:-,,} == "yes" ]; then - DOCKER_NAME=${DEFAULT_BASE_NAME}_${UBUNTU_VERSION}_${COMPILER} + DOCKER_NAME=${DEFAULT_BASE_NAME}_${UBUNTU_VERSION} DOCKER_TAG=":latest" fi @@ -143,7 +137,7 @@ fi # Check if base image exists if [ ! ${CUSTOM_ID:-,,} == "yes" ] && ! docker images | awk '{print $1}'| grep -qE "^${DOCKER_NAME}$"; then echo "Building image" - BUILD_CMD="docker build $CACHE_OPTION -t ${DOCKER_NAME} --build-arg COMPILER=$COMPILER --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} -f $BASE_DOCKERFILE ." + BUILD_CMD="docker build $CACHE_OPTION -t ${DOCKER_NAME} --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} -f $BASE_DOCKERFILE ." #echo "${BUILD_CMD}" eval "${BUILD_CMD}" else