Fall Back to Legacy 2025-03-26 OAuth Discovery for Servers Without PRM

## Motivation and Context

The MCP 2025-03-26 authorization spec predates Protected Resource Metadata:
the MCP server itself acted as the authorization base URL, its RFC 8414 metadata was fetched from
the server origin, and the spec's "Fallbacks for Servers without Metadata Discovery" section
required clients to use the default endpoints `/authorize`, `/token`, and `/register` relative to
the authorization base URL when no metadata was published. Both the TypeScript SDK
(`discoverOAuthServerInfo` falls back to the server base URL on PRM failure; `startAuthorization` /
`executeTokenRequest` / `registerClient` default the endpoint paths) and the Python SDK
(`build_oauth_authorization_server_metadata_discovery_urls(None, ...)` plus the same endpoint defaults)
keep this as client-side backwards compatibility, and the `auth/2025-03-26-oauth-metadata-backcompat` and
`auth/2025-03-26-oauth-endpoint-fallback` conformance scenarios exercise it.

The Ruby flow previously raised as soon as PRM discovery failed, so both scenarios were
expected conformance failures. This change adds the same fallback:

- `Flow#locate_authorization_server` first attempts PRM discovery; on any discovery failure
  (404s, network errors, malformed documents, matching the TypeScript and Python SDKs' broad fallback)
  the MCP server's origin becomes the legacy authorization base URL. The Communication Security check
  (HTTPS or loopback) still applies to that origin, and PRM documents that parse correctly keep
  their strict shape validation.
- `Flow#authorization_server_metadata` fetches RFC 8414 metadata from the base URL.
  On the legacy path the `issuer` byte-match is skipped: the 2025-03-26 spec predates that requirement,
  and a pre-PRM server may host its OAuth endpoints under a path prefix whose `issuer` legitimately differs
  from the discovery origin (neither reference SDK validates the issuer on this path). On the modern path
  the byte-match is unchanged.
- When even the metadata document is absent, synthetic metadata carrying the legacy spec's default endpoints
  is used, with PKCE S256 assumed (the legacy spec mandates PKCE; the TypeScript and Python SDKs hardcode S256
  on this path too). Endpoint HTTPS checks still apply.
- `run!` and `refresh!` share the new discovery helpers, and both 2025-03-26 scenarios are removed from
  `conformance/expected_failures.yml`.

## How Has This Been Tested?

New tests in `test/mcp/client/oauth/flow_test.rb`:

- the metadata-backcompat shape: no PRM, AS metadata at the server origin with `/oauth`-prefixed endpoints
  and a mismatched `issuer` completes the flow against those endpoints (no issuer error)
- the endpoint-fallback shape: no metadata at all registers, authorizes, and exchanges the code at `/register`,
  `/authorize`, and `/token` on the origin, still sending `code_challenge` (S256) and `code_verifier`
- a remote plain-http origin is rejected as the legacy authorization base
- strict-mode regression: with PRM present, a mismatched `issuer` still aborts the flow
- a malformed PRM (top-level JSON array) now selects the legacy path instead of leaking a `TypeError`,
  and surfaces a domain error when that path also dead-ends

Two existing tests that asserted PRM failure was terminal were updated to dead-end the legacy path explicitly,
preserving their original intent (single flow attempt, domain error type).

## Breaking Changes

None for spec-compliant 2025-06-18+ servers, whose PRM-based discovery and issuer validation are unchanged.
Servers that previously failed hard during PRM discovery now get one legacy discovery attempt before the flow errors,
which can only turn previously failing flows into working ones.

Author: koic

README.md

@@ -1986,6 +1986,9 @@ pass an `MCP::Client::OAuth::Provider` to the transport instead of a static `Aut
 - Send `Authorization: Bearer <access_token>` on every request when a token is available.
 - On a `401 Unauthorized`, parse the `WWW-Authenticate` header, discover the authorization server (Protected Resource Metadata + RFC 8414 Authorization Server Metadata),
   perform Dynamic Client Registration if needed, run the OAuth 2.1 Authorization Code flow with PKCE (S256), and retry the failed request with the acquired token.
+- Fall back to the legacy 2025-03-26 discovery when the server publishes no Protected Resource Metadata, matching the TypeScript and Python SDKs: the MCP server's origin acts
+  as the authorization base URL, its metadata is fetched from `<origin>/.well-known/oauth-authorization-server` without the RFC 8414 issuer byte-match (which the legacy spec predates),
+  and when even that is absent the spec's default endpoints `/authorize`, `/token`, and `/register` at the origin are used with PKCE S256 assumed.
 - On subsequent 401s with a saved `refresh_token`, exchange it at the token endpoint before falling back to the full interactive flow (RFC 6749 Section 6).
 - On a `403 Forbidden` whose `WWW-Authenticate` header carries `error="insufficient_scope"` (OAuth 2.0 step-up, RFC 6750 Section 3.1 and the MCP scope-selection-strategy),
   run a fresh authorization request for the union of the currently granted scope and the scope named in the challenge, then retry the failed request once.

conformance/expected_failures.yml

@@ -5,7 +5,5 @@ client:
   # TODO: Elicitation not implemented in Ruby client.
   - elicitation-sep1034-client-defaults
   # TODO: Remaining OAuth/auth scenarios not yet implemented in Ruby client.
-  - auth/2025-03-26-oauth-metadata-backcompat
-  - auth/2025-03-26-oauth-endpoint-fallback
   - auth/client-credentials-jwt
   - auth/cross-app-access-complete-flow

lib/mcp/client/oauth/flow.rb

@@ -38,22 +38,18 @@ def run!(server_url:, resource_metadata_url: nil, scope: nil)
             ensure_secure_url!(resource_metadata_url, label: "WWW-Authenticate resource_metadata URL")
           end
 
-          prm = fetch_protected_resource_metadata(
+          prm, authorization_server = locate_authorization_server(
             server_url: server_url,
             resource_metadata_url: resource_metadata_url,
           )
-          authorization_server = first_authorization_server(prm)
-          ensure_secure_url!(authorization_server, label: "PRM `authorization_servers` entry")
 
           # Per RFC 8707 + MCP authorization, the canonical MCP server URI is sent on
           # both the authorization and token requests. When PRM advertises a `resource`,
           # it MUST identify the same MCP server we are talking to; otherwise we are
           # being redirected to credentials minted for a different audience.
-          resource = canonical_resource(server_url: server_url, prm_resource: prm["resource"])
+          resource = canonical_resource(server_url: server_url, prm_resource: prm&.dig("resource"))
 
-          as_metadata = fetch_authorization_server_metadata(issuer_url: authorization_server)
-          ensure_issuer_matches!(expected: authorization_server, returned: as_metadata["issuer"])
-          ensure_secure_endpoints!(as_metadata)
+          as_metadata = authorization_server_metadata(authorization_server: authorization_server, legacy: prm.nil?)
 
           if provider_authorization_flow == :client_credentials
             return run_client_credentials!(as_metadata: as_metadata, prm: prm, resource: resource, scope: scope)
@@ -63,7 +59,7 @@ def run!(server_url:, resource_metadata_url: nil, scope: nil)
 
           client_info = ensure_client_registered(as_metadata: as_metadata)
 
-          effective_scope = resolve_scope(scope: scope, prm: prm)
+          effective_scope = resolve_scope(scope: scope, prm: prm || {})
           effective_scope = normalize_offline_access_scope(effective_scope, as_metadata: as_metadata)
           pkce = PKCE.generate
           state = SecureRandom.urlsafe_base64(32)
@@ -158,18 +154,14 @@ def refresh!(server_url:, resource_metadata_url: nil)
             ensure_secure_url!(resource_metadata_url, label: "WWW-Authenticate resource_metadata URL")
           end
 
-          prm = fetch_protected_resource_metadata(
+          prm, authorization_server = locate_authorization_server(
             server_url: server_url,
             resource_metadata_url: resource_metadata_url,
           )
-          authorization_server = first_authorization_server(prm)
-          ensure_secure_url!(authorization_server, label: "PRM `authorization_servers` entry")
 
-          resource = canonical_resource(server_url: server_url, prm_resource: prm["resource"])
+          resource = canonical_resource(server_url: server_url, prm_resource: prm&.dig("resource"))
 
-          as_metadata = fetch_authorization_server_metadata(issuer_url: authorization_server)
-          ensure_issuer_matches!(expected: authorization_server, returned: as_metadata["issuer"])
-          ensure_secure_endpoints!(as_metadata)
+          as_metadata = authorization_server_metadata(authorization_server: authorization_server, legacy: prm.nil?)
 
           client_info = if have_stored_client_info
             # Pre-registered / DCR-issued `client_information` always wins: if the user picked an explicit identity,
@@ -221,6 +213,87 @@ def fetch_protected_resource_metadata(server_url:, resource_metadata_url:)
           fetch_metadata_json(urls, label: "protected resource metadata")
         end
 
+        # Locates the authorization server for `server_url` and returns `[prm, authorization_server]`.
+        #
+        # Modern path (2025-06-18+): Protected Resource Metadata names the authorization server in
+        # `authorization_servers`.
+        #
+        # Legacy path (2025-03-26 backwards compatibility): when the server publishes no PRM, `prm` is nil
+        # and the MCP server's own origin acts as the authorization base URL, matching the TypeScript and Python SDKs.
+        # Any PRM discovery failure (404s, network errors, malformed documents) selects the legacy path, mirroring both SDKs' behavior.
+        # https://modelcontextprotocol.io/specification/2025-03-26/basic/authorization#fallbacks-for-servers-without-metadata-discovery
+        def locate_authorization_server(server_url:, resource_metadata_url:)
+          prm = begin
+            fetch_protected_resource_metadata(
+              server_url: server_url,
+              resource_metadata_url: resource_metadata_url,
+            )
+          rescue AuthorizationError
+            nil
+          end
+
+          if prm
+            authorization_server = first_authorization_server(prm)
+            ensure_secure_url!(authorization_server, label: "PRM `authorization_servers` entry")
+            [prm, authorization_server]
+          else
+            authorization_base = server_origin!(server_url)
+            ensure_secure_url!(authorization_base, label: "MCP server origin (legacy authorization base URL)")
+            [nil, authorization_base]
+          end
+        end
+
+        # Fetches and validates the authorization server's RFC 8414 metadata.
+        #
+        # On the modern path the metadata `issuer` must be byte-identical to the discovery URL (RFC 8414 Section 3.3).
+        # On the legacy 2025-03-26 path that validation is skipped: the legacy spec predates the requirement,
+        # and a pre-PRM server may host its OAuth endpoints under a path prefix whose `issuer` legitimately differs from
+        # the origin the metadata was discovered at (neither the TypeScript nor the Python SDK validates the issuer on this path).
+        # When even the metadata document is absent, the legacy spec's default endpoints are used.
+        def authorization_server_metadata(authorization_server:, legacy:)
+          metadata = if legacy
+            begin
+              fetch_authorization_server_metadata(issuer_url: authorization_server)
+            rescue AuthorizationError
+              default_legacy_metadata(authorization_server)
+            end
+          else
+            fetch_authorization_server_metadata(issuer_url: authorization_server).tap do |fetched|
+              ensure_issuer_matches!(expected: authorization_server, returned: fetched["issuer"])
+            end
+          end
+
+          ensure_secure_endpoints!(metadata)
+          metadata
+        end
+
+        # The 2025-03-26 spec's "Fallbacks for Servers without Metadata Discovery": clients MUST use these default endpoint paths
+        # relative to the authorization base URL. PKCE S256 is assumed because the legacy spec mandates PKCE and there is no metadata
+        # to advertise it (the TypeScript and Python SDKs hardcode S256 on this path too).
+        def default_legacy_metadata(authorization_base)
+          {
+            "issuer" => authorization_base,
+            "authorization_endpoint" => "#{authorization_base}/authorize",
+            "token_endpoint" => "#{authorization_base}/token",
+            "registration_endpoint" => "#{authorization_base}/register",
+            "code_challenge_methods_supported" => ["S256"],
+          }
+        end
+
+        # Returns `scheme://host[:port]` of `server_url`, the legacy 2025-03-26 authorization base URL for servers without PRM.
+        def server_origin!(server_url)
+          uri = URI.parse(server_url.to_s)
+          unless uri.is_a?(URI::HTTP) && uri.host
+            raise AuthorizationError,
+              "Cannot derive a legacy authorization base URL from MCP server URL #{server_url.inspect}."
+          end
+
+          port_part = uri.port == uri.default_port ? "" : ":#{uri.port}"
+          "#{uri.scheme}://#{uri.host}#{port_part}"
+        rescue URI::InvalidURIError => e
+          raise AuthorizationError, "MCP server URL #{server_url.inspect} is not a valid URI: #{e.message}."
+        end
+
         def fetch_authorization_server_metadata(issuer_url:)
           urls = Discovery.authorization_server_metadata_urls(issuer_url)
           fetch_metadata_json(urls, label: "authorization server metadata")

test/mcp/client/oauth/flow_test.rb

@@ -544,14 +544,20 @@ def test_run_raises_when_prm_resource_is_malformed_uri
           assert_match(/PRM `resource`|not a valid URI/i, error.message)
         end
 
-        def test_run_raises_when_prm_is_not_a_json_object
-          # Valid JSON but the wrong shape: indexing into a top-level array
-          # would otherwise raise `TypeError` and leak out of the SDK.
+        def test_run_falls_back_to_legacy_discovery_when_prm_is_not_a_json_object
+          # Valid JSON but the wrong shape. Any PRM discovery failure selects the legacy 2025-03-26 path
+          # (matching the TypeScript and Python SDKs); here the legacy path also dead-ends, surfacing
+          # a domain error rather than a raw `TypeError` from indexing the array.
           stub_request(:get, @prm_url).to_return(
             status: 200,
             headers: { "Content-Type" => "application/json" },
             body: "[]",
           )
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-protected-resource/mcp").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-protected-resource").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-authorization-server").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/openid-configuration").to_return(status: 404)
+          stub_request(:post, "https://srv.example.com/register").to_return(status: 404)
 
           provider = Provider.new(
             client_metadata: {
@@ -569,7 +575,147 @@ def test_run_raises_when_prm_is_not_a_json_object
             Flow.new(provider: provider).run!(server_url: @server_url, resource_metadata_url: @prm_url)
           end
 
-          assert_match(/not a JSON object/i, error.message)
+          assert_match(/Dynamic client registration failed/i, error.message)
+          assert_requested(:get, "https://srv.example.com/.well-known/oauth-authorization-server")
+        end
+
+        # Builds a provider for the legacy-discovery tests, capturing the authorization URL so tests can assert
+        # which endpoint was used.
+        def build_legacy_discovery_provider(holder)
+          Provider.new(
+            client_metadata: {
+              redirect_uris: ["http://localhost:0/callback"],
+              grant_types: ["authorization_code"],
+              response_types: ["code"],
+              token_endpoint_auth_method: "none",
+            },
+            redirect_uri: "http://localhost:0/callback",
+            redirect_handler: ->(url) {
+              holder[:authorization_url] = url
+              holder[:state] = URI.decode_www_form(url.query).to_h.fetch("state")
+            },
+            callback_handler: -> { ["test-auth-code", holder[:state]] },
+          )
+        end
+
+        def stub_prm_not_found
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-protected-resource/mcp").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-protected-resource").to_return(status: 404)
+        end
+
+        def test_run_falls_back_to_server_origin_metadata_without_prm
+          # Legacy 2025-03-26 shape: no PRM, AS metadata served from the MCP server origin,
+          # OAuth endpoints under a path prefix whose `issuer` differs from the discovery origin.
+          # The legacy path must not apply the RFC 8414 issuer byte-match (the legacy spec predates it).
+          stub_prm_not_found
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-authorization-server").to_return(
+            status: 200,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(
+              issuer: "https://srv.example.com/oauth",
+              authorization_endpoint: "https://srv.example.com/oauth/authorize",
+              token_endpoint: "https://srv.example.com/oauth/token",
+              registration_endpoint: "https://srv.example.com/oauth/register",
+              response_types_supported: ["code"],
+              code_challenge_methods_supported: ["S256"],
+              token_endpoint_auth_methods_supported: ["none"],
+            ),
+          )
+          stub_request(:post, "https://srv.example.com/oauth/register").to_return(
+            status: 201,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(client_id: "legacy-client"),
+          )
+          stub_request(:post, "https://srv.example.com/oauth/token").to_return(
+            status: 200,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(access_token: "legacy-token", token_type: "Bearer", expires_in: 3600),
+          )
+
+          holder = {}
+          provider = build_legacy_discovery_provider(holder)
+
+          result = Flow.new(provider: provider).run!(server_url: @server_url)
+
+          assert_equal(:authorized, result)
+          assert_equal("legacy-token", provider.access_token)
+          assert_equal("/oauth/authorize", holder[:authorization_url].path)
+          assert_requested(:post, "https://srv.example.com/oauth/register")
+          assert_requested(:post, "https://srv.example.com/oauth/token")
+        end
+
+        def test_run_falls_back_to_default_endpoints_without_any_metadata
+          # Legacy 2025-03-26 "Fallbacks for Servers without Metadata Discovery": with no PRM and no AS metadata,
+          # the client MUST use /authorize, /token, and /register at the authorization base URL, still sending PKCE S256.
+          stub_prm_not_found
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-authorization-server").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/openid-configuration").to_return(status: 404)
+          stub_request(:post, "https://srv.example.com/register").to_return(
+            status: 201,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(client_id: "legacy-client"),
+          )
+          stub_request(:post, "https://srv.example.com/token").to_return(
+            status: 200,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(access_token: "legacy-token", token_type: "Bearer", expires_in: 3600),
+          )
+
+          holder = {}
+          provider = build_legacy_discovery_provider(holder)
+
+          result = Flow.new(provider: provider).run!(server_url: @server_url)
+
+          assert_equal(:authorized, result)
+          assert_equal("/authorize", holder[:authorization_url].path)
+          query = URI.decode_www_form(holder[:authorization_url].query).to_h
+          assert_equal("S256", query["code_challenge_method"])
+          refute_empty(query["code_challenge"].to_s)
+          assert_requested(:post, "https://srv.example.com/register")
+          assert_requested(:post, "https://srv.example.com/token") do |req|
+            URI.decode_www_form(req.body).to_h["code_verifier"].to_s != ""
+          end
+        end
+
+        def test_run_legacy_fallback_rejects_insecure_authorization_base
+          # The Communication Security requirement still applies on the legacy path: a remote plain-http origin must not
+          # become the authorization base URL.
+          stub_request(:get, "http://internal.example.com/.well-known/oauth-protected-resource/mcp").to_return(status: 404)
+          stub_request(:get, "http://internal.example.com/.well-known/oauth-protected-resource").to_return(status: 404)
+
+          holder = {}
+          provider = build_legacy_discovery_provider(holder)
+
+          error = assert_raises(Flow::AuthorizationError) do
+            Flow.new(provider: provider).run!(server_url: "http://internal.example.com/mcp")
+          end
+
+          assert_match(/legacy authorization base URL/, error.message)
+        end
+
+        def test_run_keeps_strict_issuer_validation_when_prm_is_present
+          # The legacy issuer-check relaxation must not leak into the modern path: with PRM present,
+          # a mismatched issuer still aborts.
+          stub_request(:get, @as_metadata_url).to_return(
+            status: 200,
+            headers: { "Content-Type" => "application/json" },
+            body: JSON.generate(
+              issuer: "https://evil.example.com",
+              authorization_endpoint: "#{@auth_base}/authorize",
+              token_endpoint: "#{@auth_base}/token",
+              registration_endpoint: "#{@auth_base}/register",
+              code_challenge_methods_supported: ["S256"],
+            ),
+          )
+
+          holder = {}
+          provider = build_legacy_discovery_provider(holder)
+
+          error = assert_raises(Flow::AuthorizationError) do
+            Flow.new(provider: provider).run!(server_url: @server_url, resource_metadata_url: @prm_url)
+          end
+
+          assert_match(/`issuer` does not match/, error.message)
         end
 
         def test_run_raises_when_prm_authorization_servers_is_not_an_array

test/mcp/client/oauth/http_oauth_test.rb

@@ -1304,6 +1304,11 @@ def test_send_request_does_not_loop_when_oauth_flow_fails
             body: "",
           )
 
+          # With no PRM, discovery falls back to the legacy 2025-03-26 path; dead-end that too so the flow fails exactly once.
+          stub_request(:get, "https://srv.example.com/.well-known/oauth-authorization-server").to_return(status: 404)
+          stub_request(:get, "https://srv.example.com/.well-known/openid-configuration").to_return(status: 404)
+          stub_request(:post, "https://srv.example.com/register").to_return(status: 404)
+
           provider = Provider.new(
             client_metadata: { redirect_uris: ["http://localhost:0/callback"] },
             redirect_uri: "http://localhost:0/callback",