Skip to content

Fix empty Text event emitted with trim_text_end (#984) - #997

Draft
dralley wants to merge 3 commits into
tafia:masterfrom
dralley:empty-text2
Draft

Fix empty Text event emitted with trim_text_end (#984)#997
dralley wants to merge 3 commits into
tafia:masterfrom
dralley:empty-text2

Conversation

@dralley

@dralley dralley commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

When trim_text_end is enabled, whitespace-only text before markup or a reference is now suppressed instead of producing an empty Text("") event.

closes #984

Comment thread src/reader/mod.rs
$self.config_mut().trim_text_start = trim;
let config = $self.config_mut();
config.trim_text_start = trim_start;
config.trim_text_end = trim_end;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really like the clear->restore pattern, but it was already like this. Would be nice to refactor this away

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I also does not like it here... I was thinking about removing auto-trim altogether and making it an explicit method call on the Event

@dralley dralley Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I probably don't agree with that, at least in this specific case it structurally changes what comes out of the parser. If someone doesn't want to deal with whitespace-only text events at all (and I understand that completely, it's really quite annoying, I use trim for precisely that reason) manual trim wouldn't help.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have an old partial commit from 2022 which changes the configuration options slightly. I think this would make a lot more sense (but I think might be more difficult to implement now, will need to think about how to make it work)

-    /// Trims leading whitespace in Text events, skip the element if text is empty
-    pub trim_text_start: bool,
-    /// Trims trailing whitespace in Text events.
-    pub trim_text_end: bool,
+    /// Preserves whitespace-only text elements between events (e.g. indented "pretty" formatting).
+    pub preserve_indentation: bool,
+    /// Trims leading and trailing whitespace in Text events.
+    pub trim_text: bool,

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can file an issue for that if you agree.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt that we can reliable detect indentation and I think that is not the work for parser at all.

@dralley

dralley commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

replaces #993 and #986

@codecov-commenter

codecov-commenter commented Jul 31, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.21%. Comparing base (e00ae5c) to head (fd372a8).
⚠️ Report is 40 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #997      +/-   ##
==========================================
- Coverage   57.31%   56.21%   -1.10%     
==========================================
  Files          46       47       +1     
  Lines       18197    18446     +249     
==========================================
- Hits        10429    10370      -59     
- Misses       7768     8076     +308     
Flag Coverage Δ
unittests 56.21% <100.00%> (-1.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dralley

dralley commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Will resume this PR as well as 996 after 1001 merges.

Would be nice to include those, 1002, and maybe 897 (#897 (comment)) before cutting a new release.

Comment thread tests/reader-config.rs
When trim_text_end is enabled, whitespace-only text before markup or
a reference is now suppressed instead of producing an empty Text("") event.

closes tafia#984
The comment claimed `<` was consumed, but `read_text` never consumes
it, `<` is consumed later by `read_with` inside `read_until_close`.
The tests (and Reader) do not currently exhibit proper handling of XML
end-of-line normalization.
@dralley
dralley requested a review from Mingun August 15, 2026 22:32
@dralley
dralley marked this pull request as ready for review August 15, 2026 22:32

@Mingun Mingun left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this cannot be fix at low-level parser level. We cannot strip whitespaces before all entity references -- it is possible only after them will be expanded. &amp; should preserve spaces before it, but &xml; where it is expanded into <a>...</a> must not.

We also probably cannot blindly trim everything before <, because of CDATA sections, comments and processing instructions. Comments and PIs are stripped out from the output, so when them inside the text, spaces around them would be trimmed. I think, this would be unexpected for users.

Comment on lines +87 to +89
if trim_text_end && buf[start..].iter().all(|&b| is_whitespace(b)) {
return ReadTextResult::Markup(buf);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is not the correct place for trim. I would trim spaces before comments and CDATA sections. It seems totally wrong to me.

Comment on lines +102 to +104
if trim_text_end && buf[start..].iter().all(|&b| is_whitespace(b)) {
return ReadTextResult::Ref(buf);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is wrong. We do not want to trim spaces before entity references, otherwise " &amp; text" becomes "& text" instead of " & text". Compare the result with " x text" -- this will be unexpected for the user if the replacement in the text x with & suddenly eats up the spaces before this character.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, but given the existing tests pass, we probably need some additional test cases around all of this.

Comment thread src/reader/mod.rs
$self.config_mut().trim_text_start = trim;
let config = $self.config_mut();
config.trim_text_start = trim_start;
config.trim_text_end = trim_end;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt that we can reliable detect indentation and I think that is not the work for parser at all.

@dralley
dralley marked this pull request as draft August 17, 2026 02:51
@dralley

dralley commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator Author

I will split off the documentation fixes and new tests (which I'm adding) to a separate PR, then experiment with how to avoid having this code in the parser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

trim_text_end still emits an empty Text event for whitespace-only text before a tag

3 participants