Skip to content

feat: add gzip compression and increase API page size#219

Open
christopherblaisdell wants to merge 2 commits intodanielcopper:mainfrom
christopherblaisdell:feat/http-compression
Open

feat: add gzip compression and increase API page size#219
christopherblaisdell wants to merge 2 commits intodanielcopper:mainfrom
christopherblaisdell:feat/http-compression

Conversation

@christopherblaisdell
Copy link
Copy Markdown

Summary

Adds HTTP gzip compression and increases the API pagination page size from 50 to 250, reducing the number of HTTP round-trips by ~5x for large platforms.

What changed

http.py - request()

  • Sends \Accept-Encoding: gzip\ header on all GET requests
  • Transparently decompresses gzip responses before JSON parsing
  • Falls through cleanly when server doesn't compress (no behavioral change)

library.py - pagination loops

  • \_full_fetch_platform_roms()\: limit 50 -> 250
  • \_fetch_single_collection_roms()\: limit 50 -> 250
  • Dreamcast (362 ROMs) goes from 8 requests to 2 requests

test_http.py

  • \ est_uses_auth_header\: now also verifies Accept-Encoding header is sent
  • \ est_decompresses_gzip_response\: verifies gzip payloads are decoded correctly

Performance impact

Based on real Deck sync data (702 ROMs across 3 units):

  • Before: 8 HTTP requests for ROM lists (50 per page)
  • After: ~4 HTTP requests (250 per page)
  • Gzip compression reduces JSON payload size by ~70-80% for ROM metadata
  • Combined effect: fewer round-trips + smaller payloads = faster sync startup

Risk

Very low. Gzip decompression only activates when the server responds with \Content-Encoding: gzip\. If the server doesn't support compression, behavior is unchanged. Page size increase is well within RomM's API limits.

Christopher Blaisdell added 2 commits April 5, 2026 16:38
- Add Accept-Encoding: gzip header to HTTP GET requests
- Decompress gzip responses transparently in request()
- Increase pagination limit from 50 to 250 for ROM list fetches
- Reduces HTTP round-trips by ~5x for large platforms (e.g. Dreamcast
  362 ROMs: 8 requests -> 2 requests)
- Add tests for gzip decompression and Accept-Encoding header
@danielcopper
Copy link
Copy Markdown
Owner

No response since 2026-04-08 to the questions in #215 re: commit f86bf0d (which removes the game detail patch and would disable that whole feature).

Deferring this PR until the v1 release. Happy to resume review then if:

If neither happens by v1 release time, I'll likely close this and the sibling PRs (#215, #216, #218, #219). Ping me anytime to revive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants