Document chunk/message sizes in multipart upload

[Copilot]

Multipart upload chunk sizing was undocumented, making it difficult for users to understand upload behavior and performance characteristics.

Changes

README.md

• Added "Multipart Upload" section explaining automatic chunk size determination:
  • Files ≤ 512 MB: 32 MB chunks
  • Files 512 MB - ~49 GB: 5 MB chunks (S3 minimum)
  • Files > ~49 GB: Dynamically calculated (fileSize/10,000 parts, rounded to nearest MB)
• Included concrete examples across file size spectrum (100 MB → 1 TB)
• Documented 10 concurrent uploads with retry logic

client/g3cmd/upload-multipart.go

• Enhanced constant documentation explaining S3 constraints (5 MB minimum, 10,000 part maximum)
• Documented optimalChunkSize() calculation logic with examples
• Updated CLI help text to surface chunk sizing behavior
• Added explanation of ~49 GB threshold (10,000 parts × 5 MB = S3 limit)

.gitignore

• Added /bin/ to prevent committing build artifacts

Example

# Small files use larger chunks for performance
$ data-client upload-multipart --file-path=500MB.bam
# → 32 MB chunks (16 parts)

# Large files use dynamic chunks to stay under 10k parts
$ data-client upload-multipart --file-path=1TB.bam
# → 105 MB chunks (~9,987 parts)

Original prompt

Documentation the message size used in multi upload

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.