GET /scopes/{scope_id}/policy returns 500 (uncaught ScopeNotFoundError) for an unknown or not-yet-synced scope

[Kyzgor]

Describe the bug

GET /scopes/{scope_id}/policy returns HTTP 500 {"error": "Uncaught server exception"} for a scope that is not in the repository: an unknown scope, or an existing one during the create/sync window. It should be a 404.

get_scope_policy catches ScopeNotFoundError and falls back to _generate_default_scope_bundle, which does scopes.get("default") (scopes/api.py#L288). When no default scope exists, that re-raises ScopeNotFoundError uncaught, so the response is a 500. Sibling routes get_scope and refresh_scope already return 404 for an absent scope, and the OPAL client's policy fetcher already handles a 404.

To Reproduce

Against the route alone (no Docker/Redis):

from fastapi import FastAPI
from fastapi.testclient import TestClient
from fastapi_websocket_pubsub import PubSubEndpoint
from opal_common.authentication.deps import JWTAuthenticator
from opal_common.authentication.types import JWTAlgorithm
from opal_common.authentication.verifier import JWTVerifier
from opal_common.middleware import configure_middleware
from opal_server.scopes.api import init_scope_router
from opal_server.scopes.scope_repository import ScopeNotFoundError

class InMemoryScopeRepository:
    def __init__(self): self._scopes = {}
    async def get(self, sid):
        if sid in self._scopes: return self._scopes[sid]
        raise ScopeNotFoundError(sid)
    async def put(self, s): self._scopes[s.scope_id] = s
    async def all(self): return list(self._scopes.values())
    async def delete(self, sid): self._scopes.pop(sid, None)

def _auth():
    return JWTAuthenticator(JWTVerifier(public_key=None, algorithm=JWTAlgorithm.RS256,
        audience="https://api.opal.ac/v1/", issuer="https://opal.ac/"))

def test_absent_scope_policy_returns_500():
    app = FastAPI(); configure_middleware(app)
    app.include_router(init_scope_router(InMemoryScopeRepository(), _auth(), PubSubEndpoint()), prefix="/scopes")
    resp = TestClient(app, raise_server_exceptions=False).get("/scopes/documents/policy")
    assert resp.status_code == 500  # bug; should be 404

Or against docker/docker-compose-scopes-example.yml (it provisions myscope/herscope, so documents is absent), requesting a scope that was never created:

$ curl -s -o /dev/null -w '%{http_code}\n' 'http://localhost:7002/scopes/documents/policy?path=.'
500
$ curl -s 'http://localhost:7002/scopes/documents/policy?path=.'
{"error":"Uncaught server exception"}

Server log: the requested scope, then the default fallback, then the re-raise, so the response is a 500:

ScopeNotFoundError: Scope documents not found
ScopeNotFoundError: Scope default not found
ScopeNotFoundError: Scope documents not found

This also hits an existing scope during the normal create/sync window, since the OPAL client fetches /scopes/{scope_id}/policy?path=. on connect. The #779 reporter's posted log (v0.8.0) shows it returning 500 repeatedly until the scope syncs:

"GET /scopes/documents/policy?path=. HTTP/1.1" 500
"GET /scopes/documents/policy?path=. HTTP/1.1" 500
"GET /scopes/documents/policy?path=. HTTP/1.1" 200   # after sync

full server traceback

opal_server.scopes.scope_repository.ScopeNotFoundError: Scope documents not found
    raise ScopeNotFoundError(scope_id)
opal_server.scopes.scope_repository.ScopeNotFoundError: Scope default not found
    raise ScopeNotFoundError(scope_id)
opal_server.scopes.scope_repository.ScopeNotFoundError: Scope documents not found

Expected behavior

A 404 for an unknown or not-yet-synced scope, matching get_scope/refresh_scope and the client's 404 handling; _generate_default_scope_bundle should handle an absent default scope gracefully (or the route could degrade to an empty bundle). Happy to open a PR.

OPAL version

master (76f898e), via the test above and a scopes-mode Docker run; also in v0.8.0 (the #779 reporter's posted log).

Related: #779.