u = https://calendars.icloud.com/holidays/us_en-us.ics/
# This works as expected
Net::HTTP.get_response(URI.parse(u)).body
#> "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:icalendar-ruby\r\nCALSCALE...
# This yields a gzipped body
io = Down::NetHttp.open(request_url, rewindable: false)
io.gets
> "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0000\u000...
I know that Down sets Accept-Encoding to "" so the server won't send gzip, but in this case, the server ignores it.
I ended up doing this:
io = Down::NetHttp.open("https://calendars.icloud.com/holidays/us_en-us.ics/", rewindable: false)
if io.data[:headers]["Content-Encoding"].include?("gzip")
io.instance_variable_set(:@encoding, "binary")
io = Zlib::GzipReader.wrap(io)
end
Note that the @encoding change isn't strictly necessary here since the Gzip reader doesn't call gets, but I figured it's better to have it so the object is internally correct.
Since Down (or at least the NetHttp implementation) doesn't handle GZip transparently, it would be nice to see some sort of warning, or give us some way to control it (ie allow us to pass the Accept-Encoding header).
In any case, thanks for this library!
I know that Down sets Accept-Encoding to
""so the server won't send gzip, but in this case, the server ignores it.I ended up doing this:
Note that the
@encodingchange isn't strictly necessary here since the Gzip reader doesn't callgets, but I figured it's better to have it so the object is internally correct.Since Down (or at least the NetHttp implementation) doesn't handle GZip transparently, it would be nice to see some sort of warning, or give us some way to control it (ie allow us to pass the Accept-Encoding header).
In any case, thanks for this library!