objstore: enable azure multipart upload

[joechenrh]

What problem does this PR solve?

Issue Number: ref #68984

Problem Summary:

While routing dumpling CSV exports through the concurrent object-store upload
path, two multipart-writer issues surfaced in the shared pkg/objstore writers:

1. GCS leaves incomplete multipart uploads on failure/cancel.
   GCSWriter.Close only issued the abort (cancel()) when finalize failed.
   A failed part upload or a cancelled context stored the error and returned
   early, so the initiated upload plus every already-staged part lingered as an
   incomplete MPU (billed until a bucket lifecycle rule reaps it). This is
   pre-existing shared-writer code, but the CSV export path is the first
   dumpling caller to reach it (Concurrency > 1).

2. Azure never had a concurrent writer. AzureBlobStorage.Create discarded
   WriterOption entirely, so azblobUploader staged blocks serially no matter
   the requested concurrency — while S3 (asyncWriter) and GCS (GCSWriter)
   both got concurrent implementations. On Azure the CSV path therefore lost the
   old encode/upload overlap with nothing replacing it.

What changed and how does it work?

• GCSWriter.Close now aborts the upload on every error path via a shared
  abort(cause) helper that folds any cancel failure into the original cause.
  The success path is unchanged.

• azblobUploader gains a bounded worker pool: Create reads
  WriterOption.Concurrency/PartSize and, when concurrency is requested,
  spawns workers that StageBlock in parallel. Each chunk carries a submission
  index; Close sorts staged blocks by index before CommitBlockList so the
  committed order matches file order. On error it skips the commit (Azure
  garbage-collects the uncommitted staged blocks) and surfaces the failure.
  Callers passing no option keep the original synchronous path byte-for-byte.

Check List

Tests

• Unit test

  • Added TestAzblobConcurrentUpload (writes multi-block data via the
    concurrent path, reads back, asserts byte-identity → covers block
    ordering + integrity). It is azurite-guarded and skips when azurite is
    not running, matching the existing TestAzblob suite.
  • The GCS abort fix has no offline harness: gcs.go bypasses
    GCSWriter for the 127.0.0.1 mock endpoint and fake-gcs-server does not
    speak the XML multipart API, so GCSWriter "requires a real testing
    environment on Google Cloud" (existing repo note). It is verified by code
    reasoning here; real-GCS verification is pending.

• Manual test

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors

Release note

Fix an issue where a failed or cancelled Google Cloud Storage upload could leave an incomplete multipart upload behind.

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 66fc688e-0e7e-45cb-8d71-2b58fda8aca8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign benjamin2037 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• pkg/objstore/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-tests-checked label, please finished the tests then check the finished items in description.

For example:

Tests <!-- At least one of them must be included. -->

- [x] Unit test
- [ ] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No code

‼️ Must keep the HTML comments <!-- At least one of them must be included. -->

_{📖 For more info, you can check the "Contribute Code" section in the development guide.}