feat: Add Middleware for HTTP-request compression#6291
feat: Add Middleware for HTTP-request compression#6291davidkna-sap wants to merge 15 commits intomainfrom
Conversation
47dc29e to
828bbee
Compare
| * - If `true` is provided, the payload will always be compressed. | ||
| * - If `false` is provided, the payload will never be compressed. |
There was a problem hiding this comment.
[pp] I find it a little odd, that this is a combination of boolean and strings. Consider "always" and "never" as values. That is more explicit. If that is common practice, it's fine.
There was a problem hiding this comment.
I've renamed it to always and never. In addition, I've also renamed passthrough to header-only WDYT?
| if (mode === 'never') { | ||
| return false; | ||
| } | ||
| if (mode === 'always') { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
[pp] Minor preference. Since we are using 'always' and 'never', maybe also return here string literal instead of true false. Just a pp.
There was a problem hiding this comment.
[pp] Sorry to reopen this can... I disagree. This only makes things more complicated... This function should return a boolean, but only do what its name suggests: check if the payload should be compressed. Later, when it is used, we can then do this:
if (needsCompression || options.mode === 'header-only') {
// add header
}
if (needsCompression) {
// compressPayload
}
// call middleware chainThere was a problem hiding this comment.
[pp] Also, for better readability and brevity I would like to suggest the following flow for this function:
if (mode === "auto") {
// do the compression decision
return decision; // true or false
}
return mode === "always";| throw new ErrorWithCause( | ||
| "Could not determine payload size for 'auto' compression decision.", | ||
| e | ||
| ); |
There was a problem hiding this comment.
[pp/req] Should we throw or rather do a soft warning log and fallback to not compressing option? I find it a bit annoying if people use auto but the whole app crashes without fallback because of our implementation failed.
[q] When will Buffer.byteLength() throw?
There was a problem hiding this comment.
It will mainly throw if the input is not <string> | <Buffer> | <TypedArray> | <DataView> | <ArrayBuffer> | <SharedArrayBuffer>
There was a problem hiding this comment.
I guess other types does not make sense to be sent in the context of the RPT service?
But looking at the function name checkIfNeedsCompression() it is supposed to just check if needed and this is Cloud SDK which is supposed to support all request data types. I still feel bad if we throw something in a check function. Not a clean design I would say. Also we are technically allowing user sending for example number to remote if using never. Just want to make the responsibility of this function clear.
There was a problem hiding this comment.
I would fallback to return false in case we cannot compress.
There was a problem hiding this comment.
I guess other types does not make sense to be sent in the context of the RPT service?
For the parquet-endpoint FormData will be used too, but parquet is a binary format which can have internal compression. Compression with FormData might be per-item anyway.
I would fallback to
return falsein case we cannot compress.
The compress functions have the same <string> | <Buffer> | <TypedArray> | <DataView> | <ArrayBuffer> | <SharedArrayBuffer> types, so I moved the payload size check to also run in always-mode to catch incompatible types.
In think throwing for incompatible compression algorithms is fine.
| * - If `true` is provided, the payload will always be compressed. | ||
| * - If `false` is provided, the payload will never be compressed. |
| const currentValue = pickValueIgnoreCase( | ||
| requestConfig.headers, | ||
| 'content-encoding' | ||
| ); | ||
| const algorithmValue = getContentEncodingValue(algorithm); | ||
| const targetValue = currentValue | ||
| ? `${currentValue}, ${algorithmValue}` | ||
| : algorithmValue; | ||
| requestConfig.headers = mergeIgnoreCase(requestConfig.headers, { | ||
| 'content-encoding': targetValue | ||
| }); |
There was a problem hiding this comment.
[pp] I would again extract this to a separate function (e.g. addContentEncodingHeader()). That way the function name would replace the comment. I also think that the name currentValue is not very speaking in the context of this function, but good enough in an extracted function, that deals with the headers only.
There was a problem hiding this comment.
I tried to change this, but did not like how it ended up looking, so I opted to rename the variables here.
Co-authored-by: Marika Marszalkowski <marika.marszalkowski@sap.com>
Rename the middleware export and all imports from compressRequest() to compress() across the HTTP client package and tests. Update the changeset note to reference compress(). Keep the middleware API and options unchanged; the generic type parameter and options remain the same. Adjust unit tests to import and call compress() so all compression modes, algorithms (gzip, brotli, deflate, zstd), and payload types continue to be validated. Clarify naming to reduce confusion between request/response semantics while preserving existing runtime behavior.
marikaner
left a comment
There was a problem hiding this comment.
Only minor stuff. LGTM 😊
| if (mode === 'never') { | ||
| return false; | ||
| } | ||
| if (mode === 'always') { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
[pp] Sorry to reopen this can... I disagree. This only makes things more complicated... This function should return a boolean, but only do what its name suggests: check if the payload should be compressed. Later, when it is used, we can then do this:
if (needsCompression || options.mode === 'header-only') {
// add header
}
if (needsCompression) {
// compressPayload
}
// call middleware chain| if (mode === 'never') { | ||
| return false; | ||
| } | ||
| if (mode === 'always') { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
[pp] Also, for better readability and brevity I would like to suggest the following flow for this function:
if (mode === "auto") {
// do the compression decision
return decision; // true or false
}
return mode === "always";
Required for SAP/ai-sdk-js-backlog#470