Add new pipe convenience methods for the async readers#420
Conversation
## Motivation Often developers want to pipe the contents of an async reader into a writer. This is a common scenario in asynchronous streaming APIs. Readers are modeling sources of data where writers are modeling sinks. As an example imagine reading from a network socket and writing the contents into a file. Importantly, the entire operation needs to uphold backpressure. ## Modifications This PR adds 2 new `pipe` methods to each reader one for each writer type. Depending on the buffer ownership those methods will the pipe operation is copy free, requires a copy or even requires an intermediate allocation for copying. The method parameter labels are trying to express what the operation does. ## Result It is very easy for developers now to move elements from readers into writers.
Catfish-Man
left a comment
Catfish-Man
left a comment
There was a problem hiding this comment.
LGTM although like I said, I'd like to revisit it later
| /// mutated in place and remains usable after this operation. | ||
| /// | ||
| /// - Throws: An error originating from the read or write operations. | ||
| public consuming func pipe<Writer>( |
There was a problem hiding this comment.
The functionality here is great, but I really don't like this name. We can fix it later (i.e. it shouldn't block landing), but we should find something better. I'd love something like init(consumingContentsOf:), but I can see how that would get awkward if init should take other parameters.
There was a problem hiding this comment.
Other names I thought about where fuse(into:) or even read(into:). I am very open to changing the name but I intentionally did not put it on the writer types as neither an init nor a method because I felt it belongs on the producing type and allows nicer chaining. Again everything up for further discussion.
| var consumer = readerBuffer.consumeAll() | ||
| while let firstElement = consumer.next() { | ||
| var pending: ReadElement? = firstElement | ||
| do throws(EitherError<Writer.WriteFailure, Never>) { |
There was a problem hiding this comment.
…I can't believe I learned some Swift syntax from this PR 😂
| intermediateCapacity: Int | ||
| ) async throws(EitherError<ReadFailure, Writer.WriteFailure>) | ||
| where Writer: CallerAsyncWriter & ~Copyable & ~Escapable, Writer.WriteElement == ReadElement { | ||
| var buffer = UniqueArray<ReadElement>(minimumCapacity: intermediateCapacity) |
There was a problem hiding this comment.
Another place we need a modernized version of withUnsafeTemporaryAllocation
There was a problem hiding this comment.
Yeah the new withUnsafeTemporaryAllocation with OutputSpan gets us close but the OutputSpan is not available in an async context so we can't use it here...
2 participants
Motivation
Often developers want to pipe the contents of an async reader into a writer. This is a common scenario in asynchronous streaming APIs. Readers are modeling sources of data where writers are modeling sinks. As an example imagine reading from a network socket and writing the contents into a file. Importantly, the entire operation needs to uphold backpressure.
Modifications
This PR adds 2 new
pipemethods to each reader one for each writer type. Depending on the buffer ownership those methods will the pipe operation is copy free, requires a copy or even requires an intermediate allocation for copying. The method parameter labels are trying to express what the operation does.Result
It is very easy for developers now to move elements from readers into writers.