Skip to content

feat(effect): add HttpMiddleware.compression for gzip/deflate response compression#2151

Draft
kitlangton wants to merge 3 commits into
Effect-TS:mainfrom
kitlangton:kit/add-compression-middleware
Draft

feat(effect): add HttpMiddleware.compression for gzip/deflate response compression#2151
kitlangton wants to merge 3 commits into
Effect-TS:mainfrom
kitlangton:kit/add-compression-middleware

Conversation

@kitlangton
Copy link
Copy Markdown
Contributor

Summary

Adds an HTTP response compression middleware so Effect HttpApi users don't have to roll their own. Effect-smol's HttpMiddleware ships logging, tracing, and CORS, but no compression — Hono, Express, Fastify, and nginx all ship one.

This is platform-agnostic: it uses the Web Standard CompressionStream (available in Node 18+, Bun, Deno, and browser-style runtimes) instead of node:zlib, so it works anywhere effect-smol runs.

What's added

  • HttpMiddleware.compression(options?) — middleware that conditionally compresses response bodies with gzip or deflate.
  • HttpRouter.compression(options?) — matching Layer<never, never, HttpRouter> registration, mirroring the cors pattern.
  • HttpServerResponse.removeHeader — small public addition; the compression middleware uses it to clear stale Content-Length after swapping in a Stream body of unknown length.

Defaults (mirroring Hono's compress)

  • threshold: 1024 bytes
  • compressibleContentType: regex covering text/*, common application/* (json/xml/javascript/wasm/…), image/svg+xml, +json/+xml/+yaml/+text suffixes, etc. text/event-stream is always excluded so SSE chunks flow immediately.
  • encodings: ["gzip", "deflate"] — gzip preferred when both are accepted. br is intentionally not included; it would need separate handling and isn't supported by CompressionStream everywhere.
  • skip?: (request) => boolean — user predicate (e.g., to bypass specific paths).

Skip conditions

  • HEAD requests
  • Response already has Content-Encoding
  • Response has Cache-Control: no-transform
  • Content type not in the allowlist, or is text/event-stream
  • Known Content-Length < threshold
  • Body is Empty / Raw / FormData (only Uint8Array and Stream are compressed)

When compressing

  • Replaces body via Stream (or Uint8Array if the original body was Uint8Array)
  • Sets Content-Encoding
  • Drops stale Content-Length for stream bodies (compressed length unknown)
  • Merges Accept-Encoding into any existing Vary header (preserving e.g. Vary: Origin from CORS)
  • Weakens strong ETag to W/...

Test plan

15 new unit tests covering all branches:

  • Compresses Uint8Array body with gzip; round-trip decompresses correctly
  • Prefers gzip when both gzip and deflate are accepted
  • Compresses Stream body and clears stale Content-Length
  • Skips when Accept-Encoding is missing
  • Skips when no encoding is supported (e.g., br only)
  • Skips bodies smaller than threshold
  • Skips non-compressible content types (e.g., image/png)
  • Skips text/event-stream
  • Skips when response already has Content-Encoding
  • Skips when Cache-Control: no-transform
  • Skips HEAD requests
  • Respects user skip predicate
  • Merges Accept-Encoding into existing Vary header
  • Weakens strong ETag ("abc"W/"abc")
  • Leaves weak ETag unchanged
  • pnpm check:tsgo, pnpm lint-fix, pnpm docgen
  • All 107 test/unstable/http/ tests pass
  • Changeset added (minor)

Open questions for the maintainers

  • Naming: I went with HttpMiddleware.compression to match the existing module style; happy to rename to compress if that's preferred.
  • Brotli: skipped intentionally for a first cut; CompressionStream doesn't accept "br" in Node, so it would need a Node-only path. Easy follow-up.
  • Configuration shape: the compressibleContentType option is a single RegExp (matching Hono). Open to making it a predicate (contentType) => boolean if more flexibility is wanted.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 9, 2026

🦋 Changeset detected

Latest commit: ce75a9c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 27 packages
Name Type
effect Major
@effect/opentelemetry Major
@effect/platform-browser Major
@effect/platform-bun Major
@effect/platform-node-shared Major
@effect/platform-node Major
@effect/vitest Major
@effect/ai-anthropic Major
@effect/ai-openai-compat Major
@effect/ai-openai Major
@effect/ai-openrouter Major
@effect/atom-react Major
@effect/atom-solid Major
@effect/atom-vue Major
@effect/sql-clickhouse Major
@effect/sql-d1 Major
@effect/sql-libsql Major
@effect/sql-mssql Major
@effect/sql-mysql2 Major
@effect/sql-pg Major
@effect/sql-pglite Major
@effect/sql-sqlite-bun Major
@effect/sql-sqlite-do Major
@effect/sql-sqlite-node Major
@effect/sql-sqlite-react-native Major
@effect/sql-sqlite-wasm Major
@effect/openapi-generator Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

kitlangton added 2 commits May 9, 2026 13:53
- pickCompressionEncoding now tokenizes Accept-Encoding (no more substring
  match — 'gzipold' won't match 'gzip') and strips q-value parameters
- options.skip is now a Predicate<HttpServerRequest> for parity with cors
- HttpRouter.compression infers its options type from HttpMiddleware.compression
- Move stale Content-Length removal into the Stream branch where it's actually
  needed, instead of guarding inside applyHeaders
…on regex

The compressible content-type set is consistent with IANA mime-db's
`compressible: true` flags, so the comment now points to mime-db as the
authority. Already-compressed formats (woff/woff2, image/png, image/jpeg,
video/audio/zip) remain excluded by design.

Also:
- Bake `text/event-stream` exclusion into the regex with a negative lookahead
  so the policy is correct in isolation, not just via the runtime guard.
- Drop redundant explicit `image/svg+xml` and `application/xhtml+xml` —
  already matched by the `+xml` suffix arm.
- Add a 20-case parametrised test matrix locking the policy against mime-db,
  including the load-bearing exclusions of woff/woff2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant