Skip to content

feat: text-to-sfx, video-to-sfx and task polling (v0.2.0) - #3

Merged
spencer-zqian merged 18 commits into
mainfrom
feat/sfx
Jul 12, 2026
Merged

feat: text-to-sfx, video-to-sfx and task polling (v0.2.0)#3
spencer-zqian merged 18 commits into
mainfrom
feat/sfx

Conversation

@spencer-zqian

@spencer-zqian spencer-zqian commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Adds the three new SFX endpoints to the Python SDK, and fixes a set of bugs found along the way — several of which affect the already-shipped music endpoints too.

What's new

SFX is submit-then-poll (unlike the streaming music endpoints), so the SDK wraps the polling but keeps the primitives exposed:

with Sonilo() as client:
    result = client.text_to_sfx.generate(prompt="glass shattering", duration=5)
    result.save("sfx.m4a")
task = client.video_to_sfx.submit(
    video="clip.mp4",                       # or video_url=...
    segments=[{"start": 0, "end": 2.5, "prompt": "footsteps on gravel"}],
    audio_format="wav",
)
result = client.tasks.wait(task.task_id)    # or tasks.get() for a single poll
result.save("audio.wav")
result.save("with_audio.mp4", which="video")
  • text_to_sfx / video_to_sfx: submit() (returns SfxTask) and generate() (= submit + wait).
  • tasks: get() (single fetch, never raises on a failed task) and wait() (polls to a terminal state).
  • SfxResult.save() / asave() download the presigned R2 URLs; no API key is ever sent to the R2 host.
  • New errors: TaskFailedError (with .code, .refunded) and TaskTimeoutError (resumable — the task keeps running server-side).
  • AsyncSonilo mirrors the whole surface.

Bug fixes (several pre-date this branch)

The SDK was parsing an error field the API never sends. The public /v1/* routes return {"code", "message"} (see backend/app/errors.py), not FastAPI's {"detail"} — so every API error degraded to HTTP 400: Bad Request and the real cause was discarded. This affected the shipped music endpoints too. APIError now exposes .code and .errors (422 validation details), and BadRequestError.detail works again.

Also fixed, all with regression tests:

  • Silent audio corruption: base64.b64decode() is lenient by default — it discards invalid characters and re-aligns, so a corrupted audio_chunk decoded to wrong, shorter bytes and generate() returned a "successful" Track with silently truncated audio. Now decodes strictly, matching the JS SDK's atob() semantics (WHATWG forgiving-base64) on all 21 payloads cross-checked.
  • A malformed audio_chunk (missing data) was silently dropped as an unknown event, likewise yielding empty audio with no error. Now raises GenerationError.
  • A bare null NDJSON line crashed with an untyped AttributeError.
  • Malformed task bodies raised a bare KeyError, bypassing callers' except SoniloError.
  • SfxResult.save()/asave() used httpx's 5-second default timeout — any real download with a >5s stall failed. Now 600s, overridable.
  • tasks.get() didn't URL-encode the task id.
  • wait() slept the full poll interval past its own deadline, and sync/async disagreed on a negative poll_interval (one raised, the other busy-polled). Both now validate and clamp.

Verification

pytest: 126 passing.

End-to-end against production: text-to-sfx and video-to-sfx (multipart upload + segments) both generate, poll, download and save real audio/video — downloaded bytes match the reported file_size, WAV/MP3/MP4 headers validate, and the muxed video carries the generated audio track. Error mapping, validation, 404s and path-injection guards verified live.

Versions bumped 0.1.0 → 0.2.0. The mirrored JS SDK change is sonilo-ai/sonilo-js#1.

Percent-encode task_id before interpolating it into the /v1/tasks/{id}
path so ids containing path separators or other reserved characters
can't alter the request path.
Document TaskFailedError/TaskTimeoutError in the README error list,
add sfx/sound-effects to package keywords, assert no auth header leaks
to presigned download URLs, add an async wait() timeout test, and drop
a stale review-cycle comment.
- _raise_if_failed no longer crashes when the backend sends `error` as a
  non-dict (e.g. a bare string).
- parse_sfx_result/parse_sfx_task raise SoniloError instead of a bare
  KeyError on malformed/truncated task bodies, so callers catching
  SoniloError aren't bypassed.
- SfxResult.save()/asave() take a timeout kwarg (default 600s) instead of
  inheriting httpx's ~5s default, matching the SDK client's own timeout.
- Tasks.wait/AsyncTasks.wait now both raise SoniloError on a negative
  poll_interval or timeout, instead of diverging (sync raises ValueError
  from time.sleep, async busy-polls silently).
- _TrackBuilder.add() silently dropped any audio_chunk event whose `data`
  was missing or not a decodable base64 string (it matched no branch, so
  it fell through as an unknown event), letting collect_track()/
  acollect_track() return a successful Track with empty/truncated audio
  and no error. Now it raises GenerationError instead.
- _parse_line() called `.get()` on the result of json.loads() without
  checking it was a dict first, so a bare `null` NDJSON line crashed with
  a raw AttributeError instead of a typed error. It now treats a
  non-dict line as junk to skip, same as an empty line.
base64.b64decode() in _parse_line() was raising a raw binascii.Error
for malformed audio_chunk data before the event ever reached the
collector, so the existing malformed-chunk check in _TrackBuilder.add
never saw it and the error escaped stream()/generate() untyped,
breaking the "all errors extend SoniloError" contract.
Tasks.wait/AsyncTasks.wait slept the full poll_interval even when it
exceeded the remaining timeout, so TaskTimeoutError fired long after
the deadline (e.g. poll_interval=1000, timeout=5 slept ~1000s before
raising). Sleep is now clamped to the time remaining until deadline.
base64.b64decode(..., validate=True) required correct `=` padding and
stripped Unicode \s whitespace, unlike the JS SDK's atob(). This meant a
byte-identical audio_chunk payload (e.g. unpadded base64, or base64
containing NBSP/vertical-tab/U+2028) could decode successfully in JS
while spuriously raising GenerationError in Python.

Add _forgiving_b64decode(), which implements the WHATWG
forgiving-base64-decode algorithm atob() uses: strip only ASCII
whitespace, drop trailing padding when present, reject remainder-1
lengths and non-alphabet characters, then pad to a multiple of 4
before decoding with validate=True.

Cross-checked all 15 payloads (6 decode-success, 9 raise) against
Node's atob() in sonilo-js -- outcomes and decoded bytes agree in
every case.
Python used truthiness, so a non-string truthy `message` (e.g. 42) was
stringified verbatim while the JS SDK fell back to the default. Both SDKs
now use only a non-empty string message and fall back otherwise.
…tail

Routes under /v1/* return errors as {"code": "...", "message": "..."}
rather than FastAPI's default {"detail": "..."}, so every APIError message
was silently degrading to the HTTP reason phrase. Build the message from
`message` (falling back to legacy `detail`), and expose `.code` and
`.errors` on APIError so callers can branch on the API's typed error code
and read 422 validation details.
@spencer-zqian
spencer-zqian force-pushed the feat/sfx branch 2 times, most recently from 8821be5 to de112b1 Compare July 12, 2026 18:20
@spencer-zqian
spencer-zqian merged commit b669cac into main Jul 12, 2026
2 checks passed
@spencer-zqian
spencer-zqian deleted the feat/sfx branch July 12, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant