Skip to content

[codex] fix string literal escapes - #925

Merged
justinjoy merged 1 commit into
mainfrom
codex/fix-string-literal-escapes
Jun 28, 2026
Merged

[codex] fix string literal escapes#925
justinjoy merged 1 commit into
mainfrom
codex/fix-string-literal-escapes

Conversation

@justinjoy

Copy link
Copy Markdown
Collaborator

Summary

Fixes #924 by teaching the parser lexer to keep scanning through supported string escapes and by decoding those escapes when materializing string token values.

Root Cause

scan_string() stopped at the first " byte without considering a preceding backslash, so an input such as "a\"b" was split early and the full program parse failed. The parser also had its own raw quote-stripping path, so token decoding had no central unescape behavior.

Changes

  • Recognize \" and \\ while scanning string literals.
  • Decode \" to " and \\ to \ in wl_parser_lexer_token_to_string().
  • Reuse lexer string decoding from parser AST string value construction.
  • Add lexer and parser regression coverage for escaped quotes and backslashes.

Validation

  • meson setup build-codex --wipe -Dtests=true
  • meson test -C build-codex wirelog:lexer wirelog:parser wirelog:string_parser
  • git diff --cached --check

@justinjoy
justinjoy marked this pull request as ready for review June 28, 2026 09:23
@justinjoy
justinjoy merged commit 3f086ac into main Jun 28, 2026
27 checks passed
@justinjoy
justinjoy deleted the codex/fix-string-literal-escapes branch June 28, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser: string literals cannot contain a double quote (\" aborts whole-program parse; no escape support)

1 participant