Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 26 additions & 14 deletions .claude/skills/compile/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@ default gcc.

```bash
docker ps \
--filter ancestor=base_ddprof_24_gcc \
--filter ancestor=base_ddprof_24_clang \
--filter ancestor=base_ddprof_24 \
--format '{{.ID}}' | head -1
```

Expand All @@ -44,8 +43,7 @@ Capture the output as `CID`. If non-empty → skip to Step 3.
Confirm an image exists:

```bash
IMG=$(docker image ls --format '{{.Repository}}' \
| grep -E '^base_ddprof_24_(gcc|clang)$' | head -1)
docker image ls -f reference=base_ddprof_24 -q
```

If no image is present, stop and ask the user to run
Expand All @@ -61,7 +59,7 @@ CID=$(docker run -d --rm \
--network=host -w /app \
--cap-add CAP_SYS_PTRACE --cap-add SYS_ADMIN \
-v "$PWD:/app" \
"$IMG" \
base_ddprof_24 \
sleep infinity)
```

Expand Down Expand Up @@ -102,20 +100,34 @@ docker exec "$CID" bash -lc '
### Backgrounding & exit codes

Long builds should run via `Bash` with `run_in_background: true` so the user
isn't blocked. **Do not pipe to `tee`** when backgrounding — `tee` swallows
the failing exit code from `ninja` and the build looks like it succeeded. If
you need the log on disk, redirect with `> /tmp/ddprof_compile.log 2>&1`
instead, or set `set -o pipefail` and check `${PIPESTATUS[0]}` explicitly.
isn't blocked. The Step-3 heredoc already runs under `set -euo pipefail`, so
both forms below correctly propagate ninja's exit code:

After completion, check the exit code reported by the task notification.
On failure, extract the first compiler/clang-tidy errors:
- `ninja > /tmp/ddprof_compile.log 2>&1` — simplest, log only
- `ninja 2>&1 | tee /tmp/ddprof_compile.log` — live output + log

(Outside a `pipefail` shell, `tee` would swallow ninja's failure — redirect
instead in that case.)

After completion, **check the exit code reported by the task notification
first.** Only inspect the log if it's non-zero. Use the pattern below to find
errors, and fall back to `tail` if nothing matches so a real failure can never look
like success:

```bash
grep -E "error:" /tmp/ddprof_compile.log | head -20
errs=$(grep -nE "FAILED:|ninja: build stopped|error:|fatal error:|undefined reference|CMake Error|No space left|Killed" \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, I like the else branch

/tmp/ddprof_compile.log | head -40)
if [ -n "$errs" ]; then
printf '%s\n' "$errs"
else
# Nothing matched a known marker — dump the tail so the agent never
# silently reports success on a non-zero exit.
tail -60 /tmp/ddprof_compile.log
fi
```

Surface those (path:line + diagnostic) to the user — do not dump the whole
log.
Surface the result (path:line + diagnostic) to the user — do not dump the
whole log on success.

## Notes

Expand Down
17 changes: 8 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,6 @@ All development happens inside Docker containers. The source tree is mounted at
# Alpine (musl — matches the release binary environment)
./tools/launch_local_build.sh -f ./app/base-env-alpine/Dockerfile

# Clang instead of GCC
./tools/launch_local_build.sh --clang

# Force rebuild the Docker image
./tools/launch_local_build.sh --clean
```
Expand All @@ -52,7 +49,7 @@ Day-to-day iteration uses the Ubuntu 24 container. Execute commands inside it:
```bash
docker exec -it <container_name> bash
# or run a single command
docker exec <container_name> bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && make -j$(nproc) ddprof"
docker exec <container_name> bash -c "cd /app/build_gcc_unknown-linux-2.39_Rel && ninja ddprof"
```

## Build system
Expand Down Expand Up @@ -90,22 +87,22 @@ source setup_env.sh
# Release build (fast, optimised, LTO)
MkBuildDir Rel
RelCMake ../
make -j$(nproc) ddprof
ninja ddprof

# Debug build (symbols, no optimisation)
MkBuildDir Deb
DebCMake ../
make -j$(nproc)
ninja

# Sanitized build (ASan + UBSan — catches memory bugs)
MkBuildDir San
SanCMake ../
make -j$(nproc)
ninja

# Thread sanitizer
MkBuildDir TSan
TSanCMake ../
make -j$(nproc)
ninja
```

### Build modes at a glance
Expand All @@ -114,8 +111,10 @@ make -j$(nproc)
|---|---|---|---|
| `Rel` | `RelCMake` | Release | Performance testing, pre-release checks |
| `Deb` | `DebCMake` | Debug | Day-to-day debugging, step-through |
| `DebTidy` | `DebTidyCMake` | Debug + clang-tidy | Lint gate (clang only, slowest) |
| `San` | `SanCMake` | SanitizedDebug | Catching memory/UB errors |
| `TSan` | `TSanCMake` | ThreadSanitizedDebug | Catching data races |
| `Cov` | `CovCMake` | Coverage | Coverage instrumentation |
| `AlpRel` | `RelCMake` | Release (Alpine) | **Release binary** — what ships to users |

### Alpine (release) builds
Expand All @@ -129,7 +128,7 @@ musl binary that runs everywhere. Use the Alpine container:
source setup_env.sh
MkBuildDir AlpRel
RelCMake -DDDPROF_STATIC=ON ../
make -j$(nproc) ddprof
ninja ddprof
```

The resulting `ddprof` binary is fully static, compatible with both glibc and
Expand Down
7 changes: 1 addition & 6 deletions app/base-env/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Using a recent compiler version and recent OS (better tooling)
# We'll implement libc version sanitization in the code itself
ARG UBUNTU_VERSION=22
ARG COMPILER="gcc"

FROM ubuntu:${UBUNTU_VERSION}.04 as base
ARG UBUNTU_VERSION
Expand Down Expand Up @@ -31,11 +30,7 @@ FROM base-${UBUNTU_VERSION} AS base-gcc
ENV CC=gcc-${GCC_VERSION}
ENV CXX=g++-${GCC_VERSION}

FROM base-${UBUNTU_VERSION} AS base-clang
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we no longer need this ? I thought that CI relies on this part. No breaking changes on CI ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this does impact CI, however we do not really use the default CC/CXX in the build. So the cleanup is trivial

--build-arg COMPILER=${COMPILER} 

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both gcc and clang are always installed in the image, COMPILER was only impacting the default one used, and in CI we always passed gcc as COMPILER and then controlled the compiler used with cmake arguments.

ENV CC=clang-${CLANG_VERSION}
ENV CXX=clang++-${CLANG_VERSION}

FROM base-${COMPILER} AS final
FROM base-gcc AS final

# Tell docker to use bash as the default
SHELL ["/bin/bash", "-c"]
Expand Down
18 changes: 16 additions & 2 deletions setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,14 @@ GetDefaultAllocatorOptions() {
}

GetDirectoryExtention() {
echo "_${DDPROF_EXTENSION_CC}_${DDPROF_EXTENSION_OS}_${1}"
# DebTidy is clang-only by design — force the clang segment of the build-dir
# name regardless of CC. The vendor extension is forced separately in
# DebTidyCMake (because CmakeWithOptions calls this with BUILD_TYPE=Debug).
local CC_SUFFIX=${DDPROF_EXTENSION_CC}
if [[ "$1" == "DebTidy" ]]; then
CC_SUFFIX=clang
fi
echo "_${CC_SUFFIX}_${DDPROF_EXTENSION_OS}_${1}"
}

COMMON_OPT="${COMPILER_SETTING} ${DEFAULT_ALLOCATOR_OPT} -DCMAKE_INSTALL_PREFIX=${DDPROF_INSTALL_PREFIX} -DCOLLATZ_INSTALL_PREFIX=${DDPROF_COLLATZ_INSTALL_PREFIX} -DBUILD_BENCHMARKS=${DDPROF_BUILD_BENCH}"
Expand Down Expand Up @@ -89,7 +96,14 @@ DebCMake() {

DebTidyCMake() {
local BUILD_TYPE=Debug
# VENDOR_EXTENSION uses BUILD_TYPE (Debug), so the DebTidy special-case in
# GetDirectoryExtention does not fire here — override locally so vendored
# deps land in _clang_*_Debug and are shared with CC=clang DebCMake builds
# rather than rebuilt for clang-tidy specifically.
local DDPROF_EXTENSION_CC=clang
if [[ ( -n "${CC:-}" && "${CC%-*}" != "clang" ) || ( -n "${CXX:-}" && "${CXX%-*}" != "clang++" ) ]]; then
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to fail early on this

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not fail, but warns that CC/CXX are ignored.

echoerr "DebTidyCMake: forcing clang/clang++ (ignoring CC=${CC:-} CXX=${CXX:-} — clang-tidy requires clang)."
fi
CmakeWithOptions ${BUILD_TYPE} -DENABLE_CLANG_TIDY=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ $@
}

Expand All @@ -109,7 +123,7 @@ CovCMake() {
}

## Build a directory with a naming that reflects the OS / compiler we are using
## Example : mkBuildDir Rel --> build_UB18_clang_Rel
## Example: MkBuildDir Rel --> build_gcc_unknown-linux-2.39_Rel
MkBuildDir() {
local BUILD_DIR_EXTENSION=$(GetDirectoryExtention ${1})
echo ${BUILD_DIR_EXTENSION}
Expand Down
14 changes: 4 additions & 10 deletions tools/launch_local_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,9 @@ usage() {
echo " Optional parameters "
echo " --dockerfile/-f : use a custom docker file."
echo " --clean/-c : rebuild the image before creating it."
echo " --ubuntu_version/-u : specify ubuntu version (expected values: 16 / 18 / 20)"
echo " --ubuntu_version/-u : specify ubuntu version (expected values: 16 / 18 / 20 / 22 / 24)"
echo " --image_id/-i : use a specified docker ID, conflicts with -u."
echo " --clang : use clang instead of gcc.
--cap-test : add CAP_SETUID/SETGID/IPC_LOCK/SETFCAP for capability unit tests."
echo " --cap-test : add CAP_SETUID/SETGID/IPC_LOCK/SETFCAP for capability unit tests."
}

if [ $# != 0 ] && [ "$1" == "-h" ]; then
Expand All @@ -46,7 +45,6 @@ fi
PERFORM_CLEAN=0
# This default is to ensure that users that compile from source are likely to have a compatible libc
UBUNTU_VERSION=18
COMPILER="gcc"
EXTRA_CAPS=""
USER_OPTION="-u $(id -u):$(id -g)"

Expand Down Expand Up @@ -75,10 +73,6 @@ while [ $# != 0 ]; do
shift
shift
;;
--clang)
COMPILER="clang"
shift
;;
--cap-test)
# setcap requires CAP_SETFCAP in the effective set, which only root
# has by default. Start as root so setcap works; use 'su <user>' to
Expand Down Expand Up @@ -124,7 +118,7 @@ fi

# If we didn't pass a custom ID, then focus on Ubuntu
if [ ! ${CUSTOM_ID:-,,} == "yes" ]; then
DOCKER_NAME=${DEFAULT_BASE_NAME}_${UBUNTU_VERSION}_${COMPILER}
DOCKER_NAME=${DEFAULT_BASE_NAME}_${UBUNTU_VERSION}
DOCKER_TAG=":latest"
fi

Expand All @@ -143,7 +137,7 @@ fi
# Check if base image exists
if [ ! ${CUSTOM_ID:-,,} == "yes" ] && ! docker images | awk '{print $1}'| grep -qE "^${DOCKER_NAME}$"; then
echo "Building image"
BUILD_CMD="docker build $CACHE_OPTION -t ${DOCKER_NAME} --build-arg COMPILER=$COMPILER --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} -f $BASE_DOCKERFILE ."
BUILD_CMD="docker build $CACHE_OPTION -t ${DOCKER_NAME} --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} -f $BASE_DOCKERFILE ."
#echo "${BUILD_CMD}"
eval "${BUILD_CMD}"
else
Expand Down