Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions .claude/skills/compile/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
---
name: compile
description: Build ddprof inside the Ubuntu 24 dev Docker container. Defaults to Debug (gcc, no clang-tidy) for speed. Use when the user asks to compile, build, or rebuild the project. Reuses a running container when possible, otherwise launches one in the background. Accepts an optional mode arg (Deb/Rel/DebTidy/San/TSan/Cov).
---

# Compile ddprof (Ubuntu 24 dev container)

Builds **all targets** inside the Ubuntu 24 dev container. Default mode is
**Debug** (gcc, no clang-tidy) — fastest iteration. Override via `args` when
the user wants something else.

## Args

`args` is a single optional token: the build mode.

| `args` | Mode | CC/CXX | CMake helper | Notes |
|------------|-----------|-----------|-------------------|------------------------------------------|
| (empty) | `Deb` | gcc | `DebCMake` | Default. Fastest. No clang-tidy. |
| `rel` | `Rel` | gcc | `RelCMake` | Optimised + LTO. Slow link. |
| `debtidy` | `DebTidy` | clang | `DebTidyCMake` | Enables clang-tidy. Slowest. Lint gate. |
| `san` | `San` | gcc | `SanCMake` | ASan + UBSan. |
| `tsan` | `TSan` | gcc | `TSanCMake` | ThreadSanitizer. |
| `cov` | `Cov` | gcc | `CovCMake` | Coverage instrumentation. |

Match `args` case-insensitively. If the user's natural-language request
implies a mode (e.g. "do a release build", "run with sanitizers"), prefer
that over the default. Do not set `CC`/`CXX` — `DebTidyCMake` passes the
clang compiler flags to cmake directly; all other modes use the container's
default gcc.

## Step 1 — Find a running container

```bash
docker ps \
--filter ancestor=base_ddprof_24 \
--format '{{.ID}}' | head -1
```

Capture the output as `CID`. If non-empty → skip to Step 3.

## Step 2 — Start one in the background (only if nothing is running)

Confirm an image exists:

```bash
docker image ls -f reference=base_ddprof_24 -q
```

If no image is present, stop and ask the user to run
`./tools/launch_local_build.sh -u 24` once — the first-time image build is
long and interactive. Do not try to build the image automatically.

If the image exists, start a detached container with the same mounts as the
launch script (minus the interactive TTY and SSH agent):

```bash
CID=$(docker run -d --rm \
-u "$(id -u):$(id -g)" \
--network=host -w /app \
--cap-add CAP_SYS_PTRACE --cap-add SYS_ADMIN \
-v "$PWD:/app" \
base_ddprof_24 \
sleep infinity)
```

Note: uses `sleep infinity` so subsequent `docker exec` calls have a host to
attach to. Let the user reclaim it; do not auto-kill.

## Step 3 — Run the build via `docker exec`

Substitute `<MODE>` and `<HELPER>` from the table above. For modes other than
`DebTidy`, omit the `CC`/`CXX` exports.

Example for the default (`Deb`):

```bash
docker exec "$CID" bash -lc '
set -euo pipefail
cd /app
source ./setup_env.sh
MkBuildDir Deb
DebCMake ../
ninja
'
```

Example for `DebTidy`:

```bash
docker exec "$CID" bash -lc '
set -euo pipefail
cd /app
source ./setup_env.sh
MkBuildDir DebTidy
DebTidyCMake ../
ninja
'
```

### Backgrounding & exit codes

Long builds should run via `Bash` with `run_in_background: true` so the user
isn't blocked. The Step-3 heredoc already runs under `set -euo pipefail`, so
both forms below correctly propagate ninja's exit code:

- `ninja > /tmp/ddprof_compile.log 2>&1` — simplest, log only
- `ninja 2>&1 | tee /tmp/ddprof_compile.log` — live output + log

(Outside a `pipefail` shell, `tee` would swallow ninja's failure — redirect
instead in that case.)

After completion, **check the exit code reported by the task notification
first.** Only inspect the log if it's non-zero. Use the pattern below to find
errors, and fall back to `tail` if nothing matches so a real failure can never look
like success:

```bash
errs=$(grep -nE "FAILED:|ninja: build stopped|error:|fatal error:|undefined reference|CMake Error|No space left|Killed" \
/tmp/ddprof_compile.log | head -40)
if [ -n "$errs" ]; then
printf '%s\n' "$errs"
else
# Nothing matched a known marker — dump the tail so the agent never
# silently reports success on a non-zero exit.
tail -60 /tmp/ddprof_compile.log
fi
```

Surface the result (path:line + diagnostic) to the user — do not dump the
whole log on success.

## Notes

- Build directory is derived from `MkBuildDir <suffix>` and the host
libc/compiler triple, e.g. `build_gcc_unknown-linux-2.39_Deb`,
`build_clang_unknown-linux-2.39_DebTidy`. Each mode has its own dir, so
switching modes does not clobber the previous build.
- `ninja` auto-detects core count; no `-j` flag needed.
- For Alpine/release builds, this skill is the wrong tool — see
`CLAUDE.md` § "Alpine (release) builds".
17 changes: 8 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,6 @@ All development happens inside Docker containers. The source tree is mounted at
# Alpine (musl — matches the release binary environment)
./tools/launch_local_build.sh -f ./app/base-env-alpine/Dockerfile

# Clang instead of GCC
./tools/launch_local_build.sh --clang

# Force rebuild the Docker image
./tools/launch_local_build.sh --clean
```
Expand All @@ -52,7 +49,7 @@ Day-to-day iteration uses the Ubuntu 24 container. Execute commands inside it:
```bash
docker exec -it <container_name> bash
# or run a single command
docker exec <container_name> bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && make -j$(nproc) ddprof"
docker exec <container_name> bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && ninja ddprof"
```

## Build system
Expand Down Expand Up @@ -90,22 +87,22 @@ source setup_env.sh
# Release build (fast, optimised, LTO)
MkBuildDir Rel
RelCMake ../
make -j$(nproc) ddprof
ninja ddprof

# Debug build (symbols, no optimisation)
MkBuildDir Deb
DebCMake ../
make -j$(nproc)
ninja

# Sanitized build (ASan + UBSan — catches memory bugs)
MkBuildDir San
SanCMake ../
make -j$(nproc)
ninja

# Thread sanitizer
MkBuildDir TSan
TSanCMake ../
make -j$(nproc)
ninja
```

### Build modes at a glance
Expand All @@ -114,8 +111,10 @@ make -j$(nproc)
|---|---|---|---|
| `Rel` | `RelCMake` | Release | Performance testing, pre-release checks |
| `Deb` | `DebCMake` | Debug | Day-to-day debugging, step-through |
| `DebTidy` | `DebTidyCMake` | Debug + clang-tidy | Lint gate (clang only, slowest) |
| `San` | `SanCMake` | SanitizedDebug | Catching memory/UB errors |
| `TSan` | `TSanCMake` | ThreadSanitizedDebug | Catching data races |
| `Cov` | `CovCMake` | Coverage | Coverage instrumentation |
| `AlpRel` | `RelCMake` | Release (Alpine) | **Release binary** — what ships to users |

### Alpine (release) builds
Expand All @@ -129,7 +128,7 @@ musl binary that runs everywhere. Use the Alpine container:
source setup_env.sh
MkBuildDir AlpRel
RelCMake -DDDPROF_STATIC=ON ../
make -j$(nproc) ddprof
ninja ddprof
```

The resulting `ddprof` binary is fully static, compatible with both glibc and
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
7 changes: 1 addition & 6 deletions app/base-env/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Using a recent compiler version and recent OS (better tooling)
# We'll implement libc version sanitization in the code itself
ARG UBUNTU_VERSION=22
ARG COMPILER="gcc"

FROM ubuntu:${UBUNTU_VERSION}.04 as base
ARG UBUNTU_VERSION
Expand Down Expand Up @@ -31,11 +30,7 @@ FROM base-${UBUNTU_VERSION} AS base-gcc
ENV CC=gcc-${GCC_VERSION}
ENV CXX=g++-${GCC_VERSION}

FROM base-${UBUNTU_VERSION} AS base-clang
ENV CC=clang-${CLANG_VERSION}
ENV CXX=clang++-${CLANG_VERSION}

FROM base-${COMPILER} AS final
FROM base-gcc AS final

# Tell docker to use bash as the default
SHELL ["/bin/bash", "-c"]
Expand Down
24 changes: 19 additions & 5 deletions setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,14 @@ GetDefaultAllocatorOptions() {
}

GetDirectoryExtention() {
echo "_${DDPROF_EXTENSION_CC}_${DDPROF_EXTENSION_OS}_${1}"
# DebTidy is clang-only by design — force the clang segment of the build-dir
# name regardless of CC. The vendor extension is forced separately in
# DebTidyCMake (because CmakeWithOptions calls this with BUILD_TYPE=Debug).
local CC_SUFFIX=${DDPROF_EXTENSION_CC}
if [[ "$1" == "DebTidy" ]]; then
CC_SUFFIX=clang
fi
echo "_${CC_SUFFIX}_${DDPROF_EXTENSION_OS}_${1}"
}

COMMON_OPT="${COMPILER_SETTING} ${DEFAULT_ALLOCATOR_OPT} -DCMAKE_INSTALL_PREFIX=${DDPROF_INSTALL_PREFIX} -DCOLLATZ_INSTALL_PREFIX=${DDPROF_COLLATZ_INSTALL_PREFIX} -DBUILD_BENCHMARKS=${DDPROF_BUILD_BENCH}"
Expand All @@ -71,7 +78,7 @@ CmakeWithOptions() {
shift
local VENDOR_EXTENSION=$(GetDirectoryExtention ${BUILD_TYPE})
# shellcheck disable=SC2086
cmake_cmd="cmake ${COMMON_OPT} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DVENDOR_EXTENSION=${VENDOR_EXTENSION} $@"
cmake_cmd="cmake -GNinja ${COMMON_OPT} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DVENDOR_EXTENSION=${VENDOR_EXTENSION} $@"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Examples in Agents.MD need to be adjusted to use ninja instead of make

echoerr "-------------- cmake command -------------- "
echoerr ${cmake_cmd}
eval ${cmake_cmd}
Expand All @@ -87,10 +94,17 @@ DebCMake() {
CmakeWithOptions ${BUILD_TYPE} $@
}

# Requires clang as compiler
DebTidyCMake() {
local BUILD_TYPE=Debug
CmakeWithOptions ${BUILD_TYPE} -DENABLE_CLANG_TIDY=ON $@
# VENDOR_EXTENSION uses BUILD_TYPE (Debug), so the DebTidy special-case in
# GetDirectoryExtention does not fire here — override locally so vendored
# deps land in _clang_*_Debug and are shared with CC=clang DebCMake builds
# rather than rebuilt for clang-tidy specifically.
local DDPROF_EXTENSION_CC=clang
if [[ ( -n "${CC:-}" && "${CC%-*}" != "clang" ) || ( -n "${CXX:-}" && "${CXX%-*}" != "clang++" ) ]]; then
echoerr "DebTidyCMake: forcing clang/clang++ (ignoring CC=${CC:-} CXX=${CXX:-} — clang-tidy requires clang)."
fi
CmakeWithOptions ${BUILD_TYPE} -DENABLE_CLANG_TIDY=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ $@
}

SanCMake() {
Expand All @@ -109,7 +123,7 @@ CovCMake() {
}

## Build a directory with a naming that reflects the OS / compiler we are using
## Example : mkBuildDir Rel --> build_UB18_clang_Rel
## Example: MkBuildDir Rel --> build_gcc_unknown-linux-2.39_Rel
MkBuildDir() {
local BUILD_DIR_EXTENSION=$(GetDirectoryExtention ${1})
echo ${BUILD_DIR_EXTENSION}
Expand Down
15 changes: 9 additions & 6 deletions test/pthread_deadlock.cc
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,26 @@ void getattr_loop(bool loop) {
} // namespace

int main() {
using namespace std::chrono_literals;
constexpr auto kStartupDelay = 10ms;
constexpr auto kRaceWindow = 100ms;
constexpr uint32_t kStopTimeoutMs = 1000;

// Start a thread that calls pthread_getattr_np in a tight loop
std::thread t(getattr_loop, true);

// Give the thread time to start running
std::this_thread::sleep_for(std::chrono::milliseconds(10));
std::this_thread::sleep_for(kStartupDelay);

// Start profiling while the thread is actively calling pthread_getattr_np
int ret = ddprof_start_profiling();
int const ret = ddprof_start_profiling();
if (ret != 0) {
fprintf(stderr, "Failed to start profiling (ret=%d)\n", ret);
return 1;
}

// Let it run for a bit to exercise the race
std::this_thread::sleep_for(std::chrono::milliseconds(100));
std::this_thread::sleep_for(kRaceWindow);

// Stop the thread
g_stop.store(true, std::memory_order_relaxed);
Expand All @@ -42,9 +47,7 @@ int main() {
std::thread t2(getattr_loop, false);
t2.join();

if (ret == 0) {
ddprof_stop_profiling(1000);
}
ddprof_stop_profiling(kStopTimeoutMs);

return 0;
}
11 changes: 6 additions & 5 deletions test/pthread_deadlock_mmap.cc
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,18 @@
// (without malloc hook being called because statically linked)
// the TLS state is not initialized (otherwise it would deadlock).
int main() {
constexpr size_t kLargeAllocBytes = size_t{1024} * 1024 * 16;
constexpr uint32_t kStopTimeoutMs = 1000;

std::latch l(1);
std::thread t([&] {
l.wait();
// large allocation to exercise the mmap hooks path.
void *p = malloc(1024 * 1024 * 16);
void *p = malloc(kLargeAllocBytes);
free(p);
});

int ret = ddprof_start_profiling();
int const ret = ddprof_start_profiling();
if (ret != 0) {
fprintf(stderr, "Failed to start profiling (ret=%d)\n", ret);
return 1;
Expand All @@ -25,9 +28,7 @@ int main() {
l.count_down();
t.join();

if (ret == 0) {
ddprof_stop_profiling(1000);
}
ddprof_stop_profiling(kStopTimeoutMs);

return 0;
}
Loading