Fix empty Text event emitted with trim_text_end (#984) - #997
Conversation
| $self.config_mut().trim_text_start = trim; | ||
| let config = $self.config_mut(); | ||
| config.trim_text_start = trim_start; | ||
| config.trim_text_end = trim_end; |
There was a problem hiding this comment.
I don't really like the clear->restore pattern, but it was already like this. Would be nice to refactor this away
There was a problem hiding this comment.
Yes, I also does not like it here... I was thinking about removing auto-trim altogether and making it an explicit method call on the Event
There was a problem hiding this comment.
I probably don't agree with that, at least in this specific case it structurally changes what comes out of the parser. If someone doesn't want to deal with whitespace-only text events at all (and I understand that completely, it's really quite annoying, I use trim for precisely that reason) manual trim wouldn't help.
There was a problem hiding this comment.
I have an old partial commit from 2022 which changes the configuration options slightly. I think this would make a lot more sense (but I think might be more difficult to implement now, will need to think about how to make it work)
- /// Trims leading whitespace in Text events, skip the element if text is empty
- pub trim_text_start: bool,
- /// Trims trailing whitespace in Text events.
- pub trim_text_end: bool,
+ /// Preserves whitespace-only text elements between events (e.g. indented "pretty" formatting).
+ pub preserve_indentation: bool,
+ /// Trims leading and trailing whitespace in Text events.
+ pub trim_text: bool,There was a problem hiding this comment.
I can file an issue for that if you agree.
There was a problem hiding this comment.
I doubt that we can reliable detect indentation and I think that is not the work for parser at all.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #997 +/- ##
==========================================
- Coverage 57.31% 56.21% -1.10%
==========================================
Files 46 47 +1
Lines 18197 18446 +249
==========================================
- Hits 10429 10370 -59
- Misses 7768 8076 +308
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Will resume this PR as well as 996 after 1001 merges. Would be nice to include those, 1002, and maybe 897 (#897 (comment)) before cutting a new release. |
When trim_text_end is enabled, whitespace-only text before markup or
a reference is now suppressed instead of producing an empty Text("") event.
closes tafia#984
The comment claimed `<` was consumed, but `read_text` never consumes it, `<` is consumed later by `read_with` inside `read_until_close`.
The tests (and Reader) do not currently exhibit proper handling of XML end-of-line normalization.
Mingun
left a comment
There was a problem hiding this comment.
Unfortunately, this cannot be fix at low-level parser level. We cannot strip whitespaces before all entity references -- it is possible only after them will be expanded. & should preserve spaces before it, but &xml; where it is expanded into <a>...</a> must not.
We also probably cannot blindly trim everything before <, because of CDATA sections, comments and processing instructions. Comments and PIs are stripped out from the output, so when them inside the text, spaces around them would be trimmed. I think, this would be unexpected for users.
| if trim_text_end && buf[start..].iter().all(|&b| is_whitespace(b)) { | ||
| return ReadTextResult::Markup(buf); | ||
| } |
There was a problem hiding this comment.
I think that this is not the correct place for trim. I would trim spaces before comments and CDATA sections. It seems totally wrong to me.
| if trim_text_end && buf[start..].iter().all(|&b| is_whitespace(b)) { | ||
| return ReadTextResult::Ref(buf); | ||
| } |
There was a problem hiding this comment.
This change is wrong. We do not want to trim spaces before entity references, otherwise " & text" becomes "& text" instead of " & text". Compare the result with " x text" -- this will be unexpected for the user if the replacement in the text x with & suddenly eats up the spaces before this character.
There was a problem hiding this comment.
Good point, but given the existing tests pass, we probably need some additional test cases around all of this.
| $self.config_mut().trim_text_start = trim; | ||
| let config = $self.config_mut(); | ||
| config.trim_text_start = trim_start; | ||
| config.trim_text_end = trim_end; |
There was a problem hiding this comment.
I doubt that we can reliable detect indentation and I think that is not the work for parser at all.
|
I will split off the documentation fixes and new tests (which I'm adding) to a separate PR, then experiment with how to avoid having this code in the parser. |
When trim_text_end is enabled, whitespace-only text before markup or a reference is now suppressed instead of producing an empty Text("") event.
closes #984