|
1 | | -# Chatbot Build Benchmark Results |
| 1 | +# Chatbot Build & Runtime Benchmark Results |
2 | 2 |
|
3 | | -Benchmark comparing C and Zig implementations of the chatbot. |
| 3 | +Benchmark comparing C (GCC and Clang, C17 and C23 standards) and Zig implementations. |
4 | 4 |
|
5 | 5 | **System:** macOS arm64 |
6 | 6 | **Date:** 2026-01-12 |
7 | 7 |
|
8 | | -## Results |
| 8 | +## Compiler Versions (Local) |
| 9 | + |
| 10 | + |
| 11 | +| Compiler | Version | |
| 12 | +| ----------- | -------------------------------------------- | |
| 13 | +| GCC (macOS) | Apple Clang 17.0.0 (gcc is aliased to clang) | |
| 14 | +| Clang | Homebrew Clang 21.1.8 | |
| 15 | +| Zig | 0.15.2 | |
| 16 | + |
| 17 | + |
| 18 | +## Runtime Results |
| 19 | + |
| 20 | + |
| 21 | +| Version | Execution Time | Relative | |
| 22 | +| --------- | -------------- | --------------- | |
| 23 | +| GCC C17 | 228ms | 2.23x | |
| 24 | +| GCC C23 | 102ms | 1.00x (fastest) | |
| 25 | +| Clang C17 | 107ms | 1.04x | |
| 26 | +| Clang C23 | 106ms | 1.03x | |
| 27 | +| Zig | 628ms | 6.15x | |
| 28 | + |
| 29 | + |
| 30 | +## Binary Sizes |
| 31 | + |
| 32 | + |
| 33 | +| Version | Size | |
| 34 | +| --------- | ---- | |
| 35 | +| GCC C17 | 33K | |
| 36 | +| GCC C23 | 33K | |
| 37 | +| Clang C17 | 33K | |
| 38 | +| Clang C23 | 33K | |
| 39 | +| Zig | 1.3M | |
9 | 40 |
|
10 | | -| Metric | C | Zig | Difference | |
11 | | -|--------|---|-----|-----------| |
12 | | -| Build Time | 93ms | 710ms | Zig is 7.63x slower | |
13 | | -| Executable Size | 33K | 1.3M | Zig is 39x larger | |
14 | 41 |
|
15 | 42 | ## Analysis |
16 | 43 |
|
17 | | -### Build Time |
18 | | -- **C (93ms)**: Fast compilation using GCC with minimal optimization |
19 | | -- **Zig (710ms)**: Longer compilation time due to Zig's more comprehensive compiler |
| 44 | +### C Compilers |
| 45 | + |
| 46 | +- **GCC C23** was fastest in this run (times vary between runs) |
| 47 | +- **Clang** produces consistently fast binaries across C17/C23 |
| 48 | +- On macOS, `gcc` is actually Apple Clang; real GCC is tested on CI (Ubuntu) |
| 49 | + |
| 50 | +### Zig |
| 51 | + |
| 52 | +- Slower due to: |
| 53 | + - GeneralPurposeAllocator overhead vs C's stack allocation |
| 54 | + - Zig 0.15 buffered I/O system overhead |
| 55 | + - Additional runtime safety checks |
| 56 | +- Much larger binary (embeds stdlib, no libc dependency) |
| 57 | + |
| 58 | +### Binary Size |
20 | 59 |
|
21 | | -The C version compiles significantly faster due to: |
22 | | -- Simpler compilation pipeline |
23 | | -- No build system overhead (direct gcc command) |
24 | | -- Minimal type checking and analysis |
| 60 | +- All C versions: 33K (links against system libc) |
| 61 | + - Zig: 1.3M (self-contained, no external dependencies) |
25 | 62 |
|
26 | | -### Executable Size |
27 | | -- **C (33K)**: Small, minimal runtime |
28 | | -- **Zig (1.3M)**: Larger due to Zig's standard library and runtime |
| 63 | +## Build Configuration |
| 64 | + |
| 65 | + |
| 66 | +| Compiler | Flags | |
| 67 | +| --------- | ---------------------------------------- | |
| 68 | +| GCC C17 | `gcc -std=c17 -Wall -Wextra -pedantic` | |
| 69 | +| GCC C23 | `gcc -std=c23 -Wall -Wextra -pedantic` | |
| 70 | +| Clang C17 | `clang -std=c17 -Wall -Wextra -pedantic` | |
| 71 | +| Clang C23 | `clang -std=c23 -Wall -Wextra -pedantic` | |
| 72 | +| Zig | `zig build` (debug mode) | |
29 | 73 |
|
30 | | -The Zig executable is larger because: |
31 | | -- Zig stdlib is embedded in the binary |
32 | | -- More comprehensive runtime features |
33 | | -- GeneralPurposeAllocator adds overhead |
34 | 74 |
|
35 | 75 | ## Notes |
36 | 76 |
|
37 | | -- Both versions are unoptimized builds |
38 | | -- C build uses `-std=c11 -Wall -Wextra -pedantic` |
39 | | -- Zig build uses default optimization level |
40 | | -- Times are from cold builds (no cache) |
| 77 | +- All versions produce identical output |
| 78 | +- C23 falls back to `-std=c2x` on older compilers |
| 79 | +- Times vary between runs; relative performance is more meaningful |
| 80 | +- CI tests both GCC and Clang on Ubuntu |
41 | 81 |
|
42 | | -## Running the Benchmark |
| 82 | +## Running the Benchmarks |
43 | 83 |
|
44 | 84 | ```bash |
| 85 | +# Full runtime benchmark (all compilers) |
| 86 | +./run_benchmarks.sh |
| 87 | + |
| 88 | +# Build-only benchmark |
45 | 89 | ./benchmark.sh |
| 90 | + |
| 91 | +# Show detected compilers |
| 92 | +cd c && make info |
46 | 93 | ``` |
47 | 94 |
|
48 | | -This will clean, rebuild both versions, and compare build times and sizes. |
|
0 commit comments