Skip to content

feat(workflows): make the voice-to-voice template real - #2499

Open
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:feat/n8n-voice-to-voice-workflow
Open

feat(workflows): make the voice-to-voice template real#2499
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:feat/n8n-voice-to-voice-workflow

Conversation

@Hoang130203

Copy link
Copy Markdown

Summary

config/n8n/05-voice-to-voice.json was a placeholder — manualTrigger +
sticky note + "connections": {} — behind a catalog card advertising "Speak,
get AI response as audio"
, "featured": true, "setupTime": "2 minutes".

It is now the full loop, on your own hardware:

Webhook POST /webhook/ods-voice  (multipart audio)
  -> Audio Attached? (IF)
       false -> Return 400
       true  -> whisper:8000      /v1/audio/transcriptions
             -> llama-server:8080 /v1/chat/completions
             -> tts:8880          /v1/audio/speech   (responseFormat: file)
             -> Return Audio      (binary)
curl -X POST http://localhost:5678/webhook/ods-voice \
  -F 'file=@question.wav' --output answer.mp3

Optional form fields: voice (default af_heart, matching the tts README),
system (override the assistant prompt), stt_model.

Every hop uses the manifest's in-network port, not the published one, so
neither the audio nor the transcript ever leaves ods-network.

Three details this needed to actually work

  • The TTS response is binary. The node sets
    options.response.response.responseFormat = "file" with
    outputPropertyName: "speech", and the responder uses
    respondWith: "binary" / responseDataSource: "set" /
    inputFieldName: "speech". Without the file format n8n would try to parse
    the MP3 as JSON and fail at the last step.
  • The system prompt is written for speech, not screen — two or three short
    sentences, no markdown, no lists, no code blocks, because the reply is read
    aloud. A default chat prompt produces bullet points that Kokoro then reads
    out as literal asterisks.
  • The upload is referenced as file, not data. n8n's handleFormData()
    names a webhook binary property after the form field key, and setting
    options.binaryPropertyName appends a counter (data0) rather than
    renaming it — see feat(workflows): make the voice transcription template real #2498 for the code path.

AI Assistance

AI assisted with drafting the node graph, the spoken-output prompt, and this
description. I verified all three endpoints and ports against their service
manifests, and confirmed the binary-response parameter shapes against the node
definitions rather than from memory.

Release Lane

  • Stable hotfix targeting release/2.6.x
  • Mainline change targeting main
  • Next-minor work targeting the next feature/minor release
  • Not sure; reviewer should help classify

Stable hotfix reason:

n/a

Changed Surface

  • Docs only
  • Tests only
  • Dashboard UI
  • Dashboard API / host agent
  • Installer / bootstrap / lifecycle
  • Docker Compose / service manifests
  • Model routing / Hermes / capabilities
  • Network exposure / auth / proxy
  • Dependencies / runtime wiring

(One JSON file under config/n8n/. An import payload for n8n; no ODS code
executes it. The catalog entry is unchanged.)

Risk And Validation

  • Risk level: Low
  • Validation run:
    • git diff --check
    • Markdown/link sanity for docs
    • Focused tests listed below
    • Dashboard lint/test/build
    • Extension audit / compose validation
    • Release-grade fleet or scoped hardware validation
    • Stable-lane patch validation, if targeting release/2.6.x

Commands/results:

# Validated against the exact node package ODS ships. compose.yaml pins
# n8nio/n8n:2.6.4; `npm view n8n@2.6.4 dependencies.n8n-nodes-base` -> 2.6.2.

$ node verify.js 05-voice-to-voice.json
n8n-nodes-base version: 2.6.2
node types loaded: 417
  checked 05-voice-to-voice.json: 8 nodes

ALL WORKFLOWS VALID

# Binary-path parameter shapes checked against the node definitions:
#   httpRequest options.response.response values
#     -> fullResponse, neverError, responseFormat[autodetect|file|json|text],
#        outputPropertyName
#   respondToWebhook respondWith options
#     -> allIncomingItems, binary, firstIncomingItem, json, jwt, noData,
#        redirect, text
#   respondToWebhook responseDataSource -> automatically | set
#   respondToWebhook inputFieldName     -> string
#
# Endpoints checked against extensions/services/*/manifest.yaml:
#   whisper      port 8000  (published 9000)
#   llama-server port 8080  (published 11434)
#   tts          port 8880

Caveat, and it is a bigger one for this workflow than the others: Docker is
not running on my dev host, so I could not run the three-hop chain end to end.
Static validation proves the file imports and that every node parameter and
version is real; it does not prove the data actually threads through three
services. This is the longest chain of the set, so it is the one I would most
want a live run on. Say the word and I will get one on a machine with Docker
before you merge.

Operational Change Check

An import payload for n8n. Nothing in the installer, compose stack, ods-cli,
or dashboard-api executes it. No existing install changes until a user imports
it.

  • This is not an operational change.
  • This is an operational change and validation is recorded above.
  • This is an operational change and validation is intentionally deferred for:

Notes For Reviewers

This is the workflow whose catalog dependencies were wrong. It declares
["whisper", "llama-server", "kokoro"], and kokoro matches no service, so
check_workflow_dependencies() silently reports it satisfied — the card says
"ready" even with TTS stopped. #2495 fixes that with a kokoro -> tts alias.
Worth landing that one first, or this workflow will look installable on a box
with no TTS and fail at the last hop.

Voice list. af_heart is the tts README's example voice. If ODS pins a
different default anywhere I did not find it, point me at it.

Part of the series making the 18 stub templates real: #2496 (chat,
code-assistant), #2497 (summarizer), #2498 (transcription). Independent files,
no overlapping lines.

config/n8n/05-voice-to-voice.json was a manualTrigger plus a sticky note
saying "Customize the nodes below to match your setup", with
"connections": {} — an empty canvas behind a catalog card advertising
"Speak, get AI response as audio".

Now the full loop:

    audio in -> whisper:8000  /v1/audio/transcriptions
             -> llama-server:8080 /v1/chat/completions
             -> tts:8880    /v1/audio/speech
             -> audio out

    curl -X POST http://localhost:5678/webhook/ods-voice \
      -F 'file=@question.wav' --output answer.mp3

Every hop is an in-network address, so neither the audio nor the
transcript leaves the machine.

Details that matter for it to actually work:

- The TTS call sets responseFormat=file with outputPropertyName=speech,
  and the responder returns that binary property. Without the file
  format n8n would try to parse the MP3 as JSON.
- The system prompt tells the model to answer in two or three spoken
  sentences with no markdown, since the reply is read aloud rather than
  displayed. Callers can override it with a `system` form field.
- The multipart upload is referenced as `file` — n8n names a webhook
  binary property after the form field key, and options.binaryPropertyName
  would produce `data0` rather than renaming it.
- `voice` defaults to af_heart, matching the tts README example.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant