Skip to content

feat: Add Middleware for HTTP-request compression#6291

Open
davidkna-sap wants to merge 15 commits intomainfrom
davidkna-sap_feat-compress-middleware
Open

feat: Add Middleware for HTTP-request compression#6291
davidkna-sap wants to merge 15 commits intomainfrom
davidkna-sap_feat-compress-middleware

Conversation

@davidkna-sap
Copy link
Member

@davidkna-sap davidkna-sap commented Jan 30, 2026

Required for SAP/ai-sdk-js-backlog#470

@davidkna-sap davidkna-sap added blocked Issues that require async communication or are blocked by external factors and removed blocked Issues that require async communication or are blocked by external factors labels Feb 6, 2026
Copy link
Contributor

@marikaner marikaner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this😊

Comment on lines 92 to 93
* - If `true` is provided, the payload will always be compressed.
* - If `false` is provided, the payload will never be compressed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] I find it a little odd, that this is a combination of boolean and strings. Consider "always" and "never" as values. That is more explicit. If that is common practice, it's fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've renamed it to always and never. In addition, I've also renamed passthrough to header-only WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good.

Comment on lines 123 to 128
if (mode === 'never') {
return false;
}
if (mode === 'always') {
return true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] Minor preference. Since we are using 'always' and 'never', maybe also return here string literal instead of true false. Just a pp.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] Sorry to reopen this can... I disagree. This only makes things more complicated... This function should return a boolean, but only do what its name suggests: check if the payload should be compressed. Later, when it is used, we can then do this:

if (needsCompression || options.mode === 'header-only') {
 // add header
}

if (needsCompression) {
  // compressPayload
}

// call middleware chain

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] Also, for better readability and brevity I would like to suggest the following flow for this function:

if (mode === "auto") {
// do the compression decision
return decision; // true or false
}

return mode === "always";

Comment on lines 135 to 138
throw new ErrorWithCause(
"Could not determine payload size for 'auto' compression decision.",
e
);
Copy link
Contributor

@ZhongpinWang ZhongpinWang Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp/req] Should we throw or rather do a soft warning log and fallback to not compressing option? I find it a bit annoying if people use auto but the whole app crashes without fallback because of our implementation failed.
[q] When will Buffer.byteLength() throw?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will mainly throw if the input is not <string> | <Buffer> | <TypedArray> | <DataView> | <ArrayBuffer> | <SharedArrayBuffer>

Copy link
Contributor

@ZhongpinWang ZhongpinWang Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess other types does not make sense to be sent in the context of the RPT service?

But looking at the function name checkIfNeedsCompression() it is supposed to just check if needed and this is Cloud SDK which is supposed to support all request data types. I still feel bad if we throw something in a check function. Not a clean design I would say. Also we are technically allowing user sending for example number to remote if using never. Just want to make the responsibility of this function clear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would fallback to return false in case we cannot compress.

Copy link
Member Author

@davidkna-sap davidkna-sap Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ZhongpinWang

I guess other types does not make sense to be sent in the context of the RPT service?

For the parquet-endpoint FormData will be used too, but parquet is a binary format which can have internal compression. Compression with FormData might be per-item anyway.

I would fallback to return false in case we cannot compress.

The compress functions have the same <string> | <Buffer> | <TypedArray> | <DataView> | <ArrayBuffer> | <SharedArrayBuffer> types, so I moved the payload size check to also run in always-mode to catch incompatible types.

In think throwing for incompatible compression algorithms is fine.

Comment on lines 92 to 93
* - If `true` is provided, the payload will always be compressed.
* - If `false` is provided, the payload will never be compressed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good.

Comment on lines 193 to 203
const currentValue = pickValueIgnoreCase(
requestConfig.headers,
'content-encoding'
);
const algorithmValue = getContentEncodingValue(algorithm);
const targetValue = currentValue
? `${currentValue}, ${algorithmValue}`
: algorithmValue;
requestConfig.headers = mergeIgnoreCase(requestConfig.headers, {
'content-encoding': targetValue
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] I would again extract this to a separate function (e.g. addContentEncodingHeader()). That way the function name would replace the comment. I also think that the name currentValue is not very speaking in the context of this function, but good enough in an extracted function, that deals with the headers only.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to change this, but did not like how it ended up looking, so I opted to rename the variables here.

davidkna-sap and others added 7 commits February 12, 2026 09:00
Co-authored-by: Marika Marszalkowski <marika.marszalkowski@sap.com>
Rename the middleware export and all imports from
compressRequest() to compress() across the HTTP client package and
tests. Update the changeset note to reference compress().

Keep the middleware API and options unchanged; the generic type
parameter and options remain the same. Adjust unit tests to import
and call compress() so all compression modes, algorithms (gzip,
brotli, deflate, zstd), and payload types continue to be validated.

Clarify naming to reduce confusion between request/response semantics
while preserving existing runtime behavior.
Copy link
Contributor

@marikaner marikaner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only minor stuff. LGTM 😊

Comment on lines 123 to 128
if (mode === 'never') {
return false;
}
if (mode === 'always') {
return true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] Sorry to reopen this can... I disagree. This only makes things more complicated... This function should return a boolean, but only do what its name suggests: check if the payload should be compressed. Later, when it is used, we can then do this:

if (needsCompression || options.mode === 'header-only') {
 // add header
}

if (needsCompression) {
  // compressPayload
}

// call middleware chain

Comment on lines 123 to 128
if (mode === 'never') {
return false;
}
if (mode === 'always') {
return true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] Also, for better readability and brevity I would like to suggest the following flow for this function:

if (mode === "auto") {
// do the compression decision
return decision; // true or false
}

return mode === "always";

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants