Skip to content

Parser: do not treat '#=>' inside string literals as assertion delimiters #3

Description

@jcouball

Summary

Fix example normalization so #=> is only treated as an assertion marker when it is an inline trailing comment or a standalone comment line, not when it appears inside Ruby string literals.

Motivation

The current normalization logic performs global string substitution:

text = text.gsub('# =>', '#=>')
text = text.gsub('#=>', "\n#=>")

This rewrites any #=> sequence regardless of Ruby syntax context. If #=> appears inside a string literal, the source code is split into invalid lines and the example fails for the wrong reason.

Concrete example

# @example
#   puts '#=> not assertion'
#   sum(1, 2) #=> 3

Current normalization produces:

["puts '", "#=> not assertion'", "sum(1,2)", "#=> 3"]

This corrupts the first line and can produce syntax errors unrelated to the documented code.

Current behaviour

normalize_example_lines blindly inserts a newline before every #=> token in the raw text, including string literal content.

Proposed solution

Parse line-by-line and treat #=> as an expectation delimiter only when one of these is true:

  1. The line starts with #=> (already split assertion line), or
  2. The token appears in a trailing Ruby comment position outside quotes.

A practical implementation is to scan each line while tracking quote state (' / ") and escapes, then split only on comment #=> boundaries that are outside quoted regions.

Scope

In scope

  • Update normalize_example_lines to be syntax-aware enough to avoid splitting inside string literals
  • Keep support for both #=> and # => assertion comment styles
  • Add integration tests showing #=> inside strings does not create assertions

Out of scope

  • Full Ruby parser integration
  • Supporting assertion markers embedded in heredoc content

Implementation notes

  1. Preserve existing behavior for valid forms:
    • sum(1, 2) #=> 3
    • sum(1, 2)\n#=> 3
    • sum(1, 2) # => 3
  2. Do not split when #=> is inside '...' or "...".
  3. Keep normalized output shape compatible with extract_expectations.

Acceptance criteria

  • puts '#=> not assertion' remains a single code line after normalization
  • sum(1, 2) #=> 3 still creates one expectation
  • sum(1, 2) # => 3 still creates one expectation
  • Existing cucumber scenarios continue to pass

Related

  • lib/yard/cli/test_examples.rbnormalize_example_lines
  • lib/yard/cli/test_examples.rbextract_expectations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions