Skip to content

llvm-test-suite.cmake.example: fix optimization level selection#17

Open
atrosinenko wants to merge 1 commit intomasterfrom
atrosinenko/fix-llvm-test-suite-cmake-example
Open

llvm-test-suite.cmake.example: fix optimization level selection#17
atrosinenko wants to merge 1 commit intomasterfrom
atrosinenko/fix-llvm-test-suite-cmake-example

Conversation

@atrosinenko
Copy link
Collaborator

@atrosinenko atrosinenko commented Feb 27, 2026

When constructing compiler command lines, the arguments in *_FLAGS_RELEASE are appended after those in *_FLAGS by CMake. Since -O3 -DNDEBUG was left intact in *_FLAGS_RELEASE, this resulted in -O3 being used instead of any custom -O<level> flag passed via CUSTOM_FLAGS.

This commit overrides both set of variables (and passes FORCE flag to set) to prevent hard-to-spot errors.

@atrosinenko atrosinenko requested a review from kovdan01 February 27, 2026 15:43
@kovdan01
Copy link
Collaborator

The *_FLAGS_RELEASE options are added after *_FLAGS ones by CMake, resulting in -O3 -DNDEBUG being added unconditionally.

@atrosinenko Could you please describe the scenario which was broken in a bit more detail? My understanding of this phrase is "If we were compiling llvm-test-suite with BUILD_TYPE=Release, but custom -Ox option in CMAKE_XXX_FLAGS, we effectively had -O3 instead of -Ox". Is this interpretation correct?

The `*_FLAGS_RELEASE` options are added after `*_FLAGS` ones by CMake,
resulting in `-O3 -DNDEBUG` being added unconditionally.

This commit overrides both set of variables (and passes `FORCE` flag
to `set`) to prevent hard-to-spot errors.
@atrosinenko atrosinenko force-pushed the atrosinenko/fix-llvm-test-suite-cmake-example branch from 8eeb9d9 to 7f5eb6b Compare February 28, 2026 10:51
@atrosinenko
Copy link
Collaborator Author

@kovdan01 I updated the description of this PR, hope the updated version is clearer.

The idea is that CUSTOM_FLAGS are intended to be a quick way to override "everything".

I use a script along these lines

opt_levels=(-O0 -O2 -Oz)
clang=/opt/llvm-pauth/bin/clang

for opt in "${opt_levels[@]}"; do
  echo "### Testing at $opt ###"
  mkdir -p /path/to/llvm-test-suite/build-tmp
  cd       /path/to/llvm-test-suite/build-tmp

  # recheck before uncommenting # rm -rf * .ninja_* .lit_test_times.txt

  cmake .. -G Ninja \
    -DCUSTOM_FLAGS="$opt" \
    -DTEST_SUITE_RUN_BENCHMARKS=ON \
    -DCMAKE_C_COMPILER="$clang" \
    -C ../cmake/caches/pauth.cmake

  ninja

  mkdir -p /tmp/llvmts-results

  /path/to/bin/llvm-lit -o "/tmp/llvmts-results/report$opt.json" . > "/tmp/llvmts-results/report$opt.log" 2>&1 || true
done

The test runs performed with different optimization levels turned out to take almost identical amount of time to complete, which seemed highly unlikely for -O0 and -O2, for example. Turned out, the reason was that I assumed it is enough to set CMAKE_C_FLAGS, CMAKE_CXX_FLAGS and so on, and then it is not actually that important whether CMAKE_BUILD_TYPE is Release, Debug or anything else, as I have already set all options explicitly. This was wrong, as the *_RELEASE (or whatever "build type" is selected) counterparts of these variables override the common options. Roughly speaking, CMake constructs command lines like this: ${CMAKE_C_COMPILER} ... ${CMAKE_C_FLAGS} ${CMAKE_C_FLAGS_RELEASE}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants