Skip to content

codecov/patch inflates uncovered-line counts for QObject headers: entire unexecuted qt_metacall regions painted red, not just real missed lines #93

Description

@Yaraslaut

Problem

scripts/coverage.sh's reporting has two different views of the same
instrumentation data that disagree sharply for QObject-derived headers,
and only one of them is what Codecov's codecov/patch check actually sees.

llvm-cov report (the plain-text table coverage.sh already prints
to the build log) shows the true, function/region-granular picture. For
example, on a recent PR (#89):

Filename                                          Regions  Missed Regions  Cover    Functions  Missed Functions  Lines  Missed Lines  Cover
examples/pastebin/gui_lib/paste_qml_bridges.hpp         2               2  0.00%            2                  2      0             0      -
examples/pastebin/include/pastebin/app/app.hpp          3               1  66.67%           3                  1      2             0    100.00%

paste_qml_bridges.hpp genuinely has 0 countable lines by this
measure (nothing in the header outside declarations), but 2 unexecuted
functions/regions — one per QObject-derived class in the file
(FormsBridge, PasteBridge). app.hpp similarly shows 0 missed
lines but 1 missed function/region (App's own meta-call trampoline).

These 1-2 unexecuted functions per class are moc's generated
qt_metacall-family dispatch code, attributed via debug info back to
the original header (where the Q_OBJECT macro that produced them
textually appears) rather than to the moc-generated .cpp. They are
never executed because nothing in this project's test suites drives
these classes through Qt's actual meta-object dispatch
(QMetaObject::invokeMethod, or a live QQmlEngine binding a
Q_INVOKABLE) — every test calls the wrapped methods directly as
ordinary C++ member functions, which is correct and sufficient for
testing the class's own logic, but never exercises the dispatch
trampoline itself. MORPH_BUILD_FORMS_QML is off in the coverage build
(confirmed in this PR's CI log: "skipping ladder_pastebin_qml and the
QML smoke test"), so there is also no live-QML-engine test in this
configuration that could exercise it even in principle.

The HTML report (llvm-cov show -format=html) and the exported
.lcov file that Codecov ingests both disagree with the report
numbers above
: they paint the entire textual span of each
unexecuted region as "0 hits" per line — including blank lines,
comments, and access specifiers (public:/private:/signals:) inside
the class body — rather than reporting the true near-zero missed-line
count llvm-cov report computes. Confirmed directly on this PR: the
HTML report showed 43 "uncovered" rows in paste_qml_bridges.hpp and 21
in app.hpp (every non-blank line of two QObject class bodies), while
llvm-cov report's own function/region-level numbers show both files at
genuinely 0 real missed lines. Codecov's codecov/patch percentage
tracks the .lcov export's line-level view, not the report command's
more accurate one — so it is scored against the inflated, whole-region
count, not the real one.

Why this matters

Every QObject-derived class across all four ladder rungs (pastebin,
bookmarks, polls, and eventually the rest) has at least one Q_OBJECT
macro, so this systematically inflates codecov/patch's uncovered-line
count for every rung's GUI-adjacent headers (gui_lib/*.hpp,
app.hpp), regardless of how thoroughly the wrapped class's actual
logic is tested — no test can "cover" a dispatch trampoline the test
suite has no reason to invoke, and the resulting patch percentage
becomes a poor, misleading proxy for real test coverage on these files.

Proposed fix

scripts/aggregate_lcov_branches.py already rewrites BRDA/BRF/BRH
records from the JSON export's "branches" data (aggregating across
per-template-instantiation duplicates — see #92 for that separate, related
class of noise). The same script (or a sibling pass) should similarly
rewrite DA: line records for QObject-derived-class regions using the
JSON export's "functions"/"regions" data (which llvm-cov export -format=lcov's own JSON side-channel already carries, and which is
exactly what llvm-cov report's more accurate numbers are computed
from) instead of the raw per-line region-paint the current .lcov
inherits unmodified. Concretely: a line inside an unexecuted region
should only count as a genuine "missed line" if it is itself an
executable statement in that region — not every line the region's
source-text span happens to enclose (comments, blank lines, access
specifiers, or any other line that isn't itself instrumented).

An alternative, narrower fix scoped just to this specific case: since
the underlying cause is Qt moc's qt_metacall-family generated code
being attributed back to the header, and that code is fundamentally
runtime-dispatch glue no unit-test-level suite is expected to exercise,
coverage.sh's SOURCES filter (or a dedicated exclusion mechanism)
could exclude the specific Q_OBJECT/moc-generated-region lines from
measurement for headers under gui_lib//app.hpp-shaped files, the
same way gui//gui_wasm/ main()-shell files are already excluded with
their own documented rationale in that script.

Not in scope here

This is a coverage-tooling/reporting-accuracy issue, not a test-coverage
gap — the actual class logic in every case cited here already has real,
CI-verified test coverage; only the meta-object dispatch trampoline
itself (which nothing in this codebase's test suites, present or
plausible, would ever legitimately drive) is unexecuted.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions