Bot review comment appears to cap changes tracked at 18 despite PR having 32 commits

[ggallen]

Bot review comment appears to have a fixed limit on changes tracked

Problem

The fullsend-ai-review[bot] comment on PR #722 (comment #4397770354) appears to be tracking a fixed number of changes (18) despite the PR having 32 total commits and continuing to receive new commits.

Observations

1. Fixed count: The "changes reviewed" count has remained at 18 across the last several review cycles
2. PR has 32 commits: git log --oneline upstream/main..HEAD | wc -l shows 32 commits in the PR branch
3. New commits not reflected: Recent commits pushed after the count reached 18 have not caused the count to increase:
   • 91752421 - Remove .gitignore change to keep PR scope focused
   • 5e7c2e60 - Add pre-existing implementation plans to README index
   • c5b40867 - Use RFC 3986 URL normalization instead of POSIX path.Clean()

Questions

1. Is there a size limit? Is the bot comment hitting GitHub's comment size limit, preventing it from listing all changes?
2. Are old reviews being lost? If there is a size limit, what happens to the older review data? Does it get truncated, or is there a rolling window?
3. Is the count accurate? Does the "18" represent:
   • The first 18 commits only?
   • A rolling window of the most recent 18 commits?
   • 18 "unique" changes (deduplicated somehow)?
   • Something else?

Expected Behavior

The review comment should either:

1. Track all changes in the PR, or
2. Clearly indicate if there's a limit (e.g., "Showing most recent 18 of 32 changes")

Impact

This makes it unclear whether the bot is actually reviewing all changes in the PR or only a subset. For long-running PRs with many commits, this could mean later changes are not being reviewed or tracked.

Reproduction

1. View PR Add implementation plan for universal harness access #722: Add implementation plan for universal harness access #722
2. Check comment #4397770354
3. Note the changes reviewed count
4. Compare to actual commit count in the PR
5. Push new commits and observe whether the count increases

Additional Context

• PR Add implementation plan for universal harness access #722 has had multiple review cycles with many commits addressing review feedback
• The PR is a documentation change (ADR + implementation plan) with no binary files or particularly large diffs
• The bot has successfully reviewed all commits individually (status checks show reviews for each SHA), so the issue appears to be specific to the tracking/display in the summary comment

[github-actions]

fullsend triage is working on this — view logs

[fullsend-ai-triage]

Triage Summary

This issue describes a display bug in the review bot's sticky summary comment. The "changes reviewed" count stops increasing at 18 despite the PR having 32 commits. Importantly, the actual reviews are still being performed for every commit (status checks confirm this), so this is purely a tracking/display issue in the summary comment.

Root Cause Hypothesis

The sticky comment system (internal/sticky/sticky.go) enforces a 65KB size limit on comment bodies. When a long-running PR accumulates many review iterations, the system collapses older content into <details> blocks and eventually drops the oldest history entries entirely. The "changes reviewed" count likely derives from entries present in the comment body rather than being tracked as an independent counter — so when old entries are truncated, the count stops growing.

Recommended Fix

The count should be maintained independently of the collapsible history content. Options include:

1. Footer-based counter: Store the total count in the comment footer (using the existing FooterMarker mechanism), so it persists across truncation cycles
2. Truncation notice: When history is dropped, display something like "Showing most recent N of M changes reviewed"
3. Both: Independent counter plus a clear truncation indicator

Proposed Test Case

Test: sticky comment preserves accurate change count across truncation

Setup:
  - Create a sticky comment manager with a low MaxSize (e.g., 2000 bytes)
  - Simulate 25+ review iterations being appended

Assertions:
  1. The total "changes reviewed" count equals the number of iterations,
     not just the number of visible (non-collapsed) entries
  2. When truncation occurs, a notice indicates history was dropped
  3. The count survives multiple collapse/truncation cycles

Edge cases:
  - Comment body exactly at the size limit boundary
  - Single review iteration with a very large body
  - Rapid successive updates (no time between review cycles)

Severity: Low

This is a cosmetic issue — all reviews are actually performed. The impact is limited to potential confusion about whether later commits in long-running PRs have been reviewed.

[github-actions]

fullsend code is working on this — view logs

[ralphbean]

/code

[github-actions]

fullsend code is working on this — view logs