Skip to content

Define the Uri parser's strictness for the path/query/fragment/userinfo character repertoire #32

Description

@OmarAlJarrah

Summary

The Uri (RFC 3986) profile validates the host against a character repertoire, but the
path, query, fragment, and userinfo components are validated against only a narrow set: C0
controls, U+007F DELETE, and malformed % triplets. Every other character that RFC 3986 forbids in
those components — including ASCII symbols that RFC 3987 also forbids — is accepted and preserved
verbatim.

This is a deliberate "preserve-by-default" posture today, but the asymmetry (strict host, liberal
everything-else) is surprising and worth an explicit decision: how strict do we want the Uri
parser to be about the character repertoire?

Trigger

Uri.parse("sa\\\\/////32e32d")  =>  Ok(value=sa\\\\/////32e32d)

Backslash (U+005C) is not part of the RFC 3986 grammar (nor RFC 3987), yet it round-trips
unchanged. It survives inside a scheme'd URI too: Uri.parse("x:sa\\/b") => Ok(value=x:sa\\/b).

Current behavior (observed on the JVM target of main)

Input (component · char) Result RFC 3986 RFC 3987
path a b (space) ACCEPT forbidden forbidden
path a"b ACCEPT forbidden forbidden
path a<b / a>b ACCEPT forbidden forbidden
path a\b (backslash) ACCEPT forbidden forbidden
path a^b ACCEPT forbidden forbidden
path a`b (backtick) ACCEPT forbidden forbidden
path a{b / a|b / a}b ACCEPT forbidden forbidden
path a[b / a]b ACCEPT forbidden forbidden
path aéb (non-ASCII) ACCEPT forbidden allowed (iunreserved)
query x<y ACCEPT forbidden forbidden
query non-ASCII ACCEPT forbidden allowed
fragment x<y / x y ACCEPT forbidden forbidden / forbidden
userinfo u<r ACCEPT forbidden forbidden
userinfo non-ASCII ACCEPT forbidden allowed
path a%20b (valid pct) ACCEPT allowed allowed
path a%zzb / lone % reject invalid invalid
host ho<st / ho st / non-ASCII reject (ForbiddenHostCodePoint) forbidden

So the host pipeline is strict; the other components are not. (a%20b is correctly accepted; bad
percent triplets are correctly rejected — those two are conformant.)

Where it comes from

UriParser validates path/query/fragment/userinfo through rawError
firstForbiddenIndex (only isC0Control() / DEL) and firstBadPercentIndex (malformed %). There
is no pchar / ipchar repertoire check on those components. This matches the documented posture in
docs/SPEC.md §8 (PARSE-49: "U+005C is an ordinary code point, never a delimiter; it is
preserved"). The host, by contrast, runs through HostParser, which rejects out-of-repertoire code
points.

Why this deserves a decision

  • The accepted ASCII symbols — space " < > \ ^ ` { | } [ ] — are outside
    both RFC 3986 pchar and RFC 3987 ipchar. There is no reading of either spec under which a
    raw < in a path is valid, so accepting them is an unambiguous deviation.
  • Raw non-ASCII (e.g. é) is allowed by RFC 3987 iunreserved but not RFC 3986. Since the profile
    is described as "RFC 3986 (3987-aware)", accepting it may be intentional — but it is inconsistent
    with the host pipeline, which rejects non-ASCII reg-names before IDNA.
  • The strict-host / liberal-rest asymmetry is the surprising part and the core of the question.

Also surfaced (secondary, not the main question)

  • Empty port dropped: Uri.parse("//h:/p") returns Ok(value=//h/p) — the empty :port is not
    preserved, which sits oddly with preserve-by-default. RFC 3986 permits an empty port (port = *DIGIT).
  • No port range cap: Uri.parse("//h:999999/p") is accepted. This is documented/intended for the
    Uri profile and is arguably RFC-conformant; noted only for completeness.

Options to discuss

  1. Keep preserve-by-default (liberal). Document explicitly that the Uri parser accepts a
    superset of the grammar in path/query/fragment/userinfo and that callers are responsible for
    encoding. Cheapest; matches the current SPEC.
  2. Strict repertoire rejection. Reject any raw code point outside pchar / ipchar in those
    components, mirroring the host pipeline. Most RFC-faithful; a breaking behavior change and needs a
    new error variant (the catalog currently folds these into InvalidPercentEncoding).
  3. Opt-in strictness via ParseOptions (e.g. strictRepertoire), defaulting to today's
    behavior. Preserves compatibility; gives conformance-sensitive callers a strict mode.
  4. Split the difference. Reject the ASCII symbols forbidden by both 3986 and 3987 (unambiguous),
    but keep accepting raw non-ASCII as the intended "3987-aware" behavior.

Seed recommendation: option 3 or 4, so the default stays compatible while a strict, conformant mode
exists.

Notes

  • If we change behavior, the SPEC (§8) and any conformance framing in the README's Standards table
    should be updated to match, and the idnaref / conformance baselines checked for impact.
  • Whatever we choose should be applied consistently across path, query, fragment, and userinfo, since
    they share the same rawError validator today.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions