Improve upload throughput on fast connections#435
Closed
got3nks wants to merge 2 commits intoamule-project:masterfrom
Closed
Improve upload throughput on fast connections#435got3nks wants to merge 2 commits intoamule-project:masterfrom
got3nks wants to merge 2 commits intoamule-project:masterfrom
Conversation
The upload throttler was designed for ADSL-era connections (~128 KiB/s) and uses conservative constants that create artificial bottlenecks on modern fast connections. This patch dynamically scales parameters based on the configured upload rate while preserving identical behavior for slow connections (< 256 KiB/s). Changes: UploadBandwidthThrottler.cpp: - Scale minFragSize/doubleSendSize dynamically with data rate for fast connections (> 256 KiB/s), removing the fixed 1300/2600 byte caps that limited per-slot data per throttler pass - Scale bandwidth banking tolerance to ~20ms worth of bandwidth, enabling efficient burst transmission instead of the fixed (slots * 512 + 1) byte cap UploadClient.cpp: - Increase per-client payload buffer from 100 KiB to 8 MiB to prevent socket starvation at high per-slot speeds - Increase packet chunk size from 10 KiB to 128 KiB in both standard and compressed packet paths, reducing per-packet header overhead and socket queue churn LibSocketAsio.cpp: - Set TCP send buffer to 512 KiB (up from OS default of ~16-64 KiB) after successful connect, allowing the kernel to batch more data per write and reducing context switches under high throughput All changes are backward compatible: - Connections below 256 KiB/s use the original constants unchanged - The two-pass fairness algorithm is preserved - No wire protocol changes; works with all stock eMule/aMule clients - Larger packets (128 KiB) are well within MAX_PACKET_SIZE (2 MB) - Socket buffer size is a local OS setting invisible to remote peers Tested on a 1 Gbps server: per-slot upload speed increased from ~450 KB/s to 6 MB/s (13x improvement) on the same hardware and connection.
Set SO_RCVBUF to 512 KiB after connect, matching the send buffer. This improves download throughput by allowing the kernel to buffer more incoming data before the application reads it.
Contributor
Author
|
Superseded by #444 — a cleaner, minimal version with just the two most impactful changes (adaptive payload buffer + adaptive chunk size), no throttler or socket layer modifications. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The upload throttler uses hardcoded constants designed for ADSL-era connections (~128 KiB/s) that create artificial bottlenecks on modern fast connections. This patch dynamically scales parameters based on the configured upload rate while preserving identical behavior for slow connections (< 256 KiB/s).
minFragSize/doubleSendSizedynamically with data rate for fast connections, removing the fixed 1300/2600 byte caps that limited per-slot data per throttler pass(slots * 512 + 1)byte capBackward compatibility
All changes are fully backward compatible:
MAX_PACKET_SIZE(2 MB)Test results
Tested on a 1 Gbps dedicated server running aMule in Docker:
Note: peak results were achieved in combination with #436 (uint16 → uint32 speed limits), which removes the 524 Mbps configuration cap. This PR alone still provides significant improvement on connections up to that limit.
Recommended configuration for fast connections
When using
MaxUpload=0(unlimited), aMule dynamically calculates the number of upload slots based on the current upload rate. This can result in a slow ramp-up where only a few slots are active initially. For best results on fast connections, set an explicitMaxUploadvalue rather than relying on unlimited mode:MaxUpload=65534(~524 Mbps, maximum before uint16 unlimited threshold)MaxUpload=100000or higher for gigabit+ connectionsThis ensures the full number of upload slots are available immediately.
Files changed
src/UploadBandwidthThrottler.cppsrc/UploadClient.cppsrc/LibSocketAsio.cpp