feat: move translation to web worker and update transformers.js#59
Open
jeremio wants to merge 40 commits intonaeruru:mainfrom
Open
feat: move translation to web worker and update transformers.js#59jeremio wants to merge 40 commits intonaeruru:mainfrom
jeremio wants to merge 40 commits intonaeruru:mainfrom
Conversation
- Moves the translation logic to a web worker to run in the browser. - Updates transformers.js to v3 to enable GPU support. - Removes the old translation worker from the Electron main process. - Fixes various TypeScript errors and configuration issues.
Owner
|
awesome!! the worker works on web side now. however it seems to be having issues using works fine with
|
* update tooltips * remove unused object data * enforce snake_case * remove unnecessary attribute * refactor * remove obsolete code * improve code readability * remove reference to nonexistent class
Contributor
Author
|
Hi @naeruru, To debug this webgpu issue, I need:
I haven't seen these errors on my Ubuntu/Chrome/GTX 1660 setup, so the browser console errors suggest webgpu compatibility issues with your specific environment. Thanks |
* update inputs of type number to v-number-input * propagate localization * improve code readability * modify typing for OSC triggers * implement migration
Owner
|
cant seem to get it to work on multiple rigs. Ive tried changing the 
steps to reproduce are simply:
|
Assigning the Promise directly (without await) ensures concurrent messages reuse the same in-flight Promise instead of starting parallel model downloads.
WebGPU is not supported on all browsers/configurations and the Xenova/nllb-200 model does not have compatible WebGPU weights in transformers.js v3. Falls back to wasm (the v2.x behaviour) on failure.
Add try/catch in the worker's onmessage handler. On failure, reset the singleton so the next attempt can retry, and send an error status back to the store. The store resets translate/loading_result flags so the log entry is no longer stuck in a permanent translating state.
The previous filter on 'onnx/encoder_model_quantized.onnx' only matched WASM downloads. Progress messages from WebGPU or other devices were silently ignored, leaving the progress bar invisible during download.
The migration only converted the global OSC port (string → number) but left per-trigger ports unconverted. Also fixes the fallback from 0 to 9000, as port 0 is invalid for OSC (valid range: 1–65535).
Translation now runs in a browser Web Worker and communicates directly with the store. The Electron IPC relay was removed in the migration to transformers.js v3 but the renderer-side listener was left behind.
The 'mimiuchi-websocketserver-close' listener registered in onMounted was never removed in onUnmounted (only 'closed' was removed, not 'close'), causing a memory leak.
The class was never exported or used anywhere. Constructor and transcribe() were empty, all recognition logic was commented out.
The plugin file was entirely commented out and never imported.
All vulnerabilities were in dev/build dependencies (vite, rollup, electron-builder, eslint). No breaking changes (semver-safe updates).
vite-env.d.ts already declares Window.ipcRenderer globally. The per-file declarations were overriding this type, defeating TypeScript's type safety for ipcRenderer calls.
Updated: @huggingface/transformers, @types/node, @vitejs/plugin-vue, electron-builder, obs-websocket-js, pinia, typescript, vite-plugin-electron, vite-plugin-vuetify, vue, vue-i18n, vue-router, vue-tsc, ws. Major packages (vuetify, electron, vite, node-osc, eslint) intentionally left for a separate dedicated update.
- Update eslint config to scope vue/block-order to .vue files only - Disable context-specific rules (process for electron/vite, self for worker) - Fix all resulting lint errors: v-bind:key on v-for, no-case-declarations, Function types, unused vars, prop camelCase, isNaN → Number.isNaN, angle-bracket type assertions, and more
All 44 icons used in the project confirmed present in v7. No renames or removals affect this codebase.
node-osc v11 ships its own TypeScript types, making @types/node-osc redundant. API is backwards-compatible: Bundle, Client, and client.send() signatures are unchanged.
API is backwards-compatible (get/set/delete, schema validation). No code changes required — project already uses ESM.
SpeechRecognitionResultList is array-like but does not inherit from Array.prototype, so .at() is unavailable. Replace with index access.
- Replace theme.global.name.value = X with theme.change(X) in Home.vue and Appearance.vue (Vuetify deprecation warning) - Fix SpeechRecognitionResultList.at(-1) → index access in WebSpeech.ts (SpeechRecognitionResultList is array-like but not an Array) The next() navigation guard warning is Vuetify 3 internal — no fix possible until Vuetify 4.
Tested: IPC (minimize/maximize/close), WebSocket server (Chrome→Electron), electron-store persistence. WebGPU→WASM translation fallback confirmed working. No API changes required.
- Remove unused stt_Settings ref from settings.ts (duplicate of speech.ts stt_init) - Remove dead language_choice ref and watcher from STT.vue (store bound directly) - Revoke object URL after export in logs.ts (memory leak on each transcript export) - Replace JSON template literals with JSON.stringify() in speech.ts (fragile construction) - Remove useConnectionsStore() self-reference in toggle_broadcast (use closure vars)
reloadEvents() was called from onUpdated(), re-registering the
receive-text-event IPC listener on every reactive update. Move
registration to onMounted() and remove the now-unused reloadEvents()
function.
Also removes the stale removeListener('websocket-connect') call which
was removing a listener that was never registered.
…elay - TTS: skip speak() on translation callback (log.isTranslationFinal) to avoid playing the transcript twice when translation is enabled - TikTok: break out of switch on fetch failure instead of retrying the same failing request (CORS/network errors are not transient) - new-line delay: pass index i as setTimeout argument to avoid logs.at(-1) resolving to the wrong log if new entries are added during the delay
stt.type was initialized as {title, value} but v-select with item-value
emits a plain string — causing the STT settings section to vanish after
any interaction. Store type as a string directly and update the template
check accordingly.
emit_osc created a new UDP Client per call and never closed it. Await
send() and close() to release the socket immediately after each message.
Multiple send-text-event IPC calls arriving while empty_queue is processing (waiting between chunks) would start concurrent instances of empty_queue on the same array, causing chunks to be sent out of order or stolen between instances. Add a queue_running flag in index.ts and an onDone callback in empty_queue() to prevent concurrent processing.
document.getElementById('loglist') in speech.ts was already broken —
the element has id="log-list" (with a hyphen). Stores should not access
the DOM.
Remove the DOM access from on_submit() and add a watch on logsStore.logs
in Home.vue that scrolls to bottom after each update using nextTick.
Internal variable name was inconsistent with its exported name.
- is_electron: drop redundant typeof userAgent check (always string) - word_replace: remove dead [] initializer overwritten in both branches - speech: remove redundant const pins aliases in pin/unpin/is_pinned_language - WebSpeech: remove unused SpeechGrammarList and SpeechRecognitionEvent fields - translation: remove onMessageReceived from store return (internal only) - electron/main: remove unused _require and createRequire import
Move clearTimeout/setTimeout logic into schedule_pause() so the timer ID is no longer part of the store's public API, eliminating the JetBrains "assign to const" false positive on logsStore.wait_interval.
ws is typed WebSocket | OBSWebSocket — after ruling out WebSocket, TypeScript already narrows to OBSWebSocket, making the else-if redundant.
jeremio
force-pushed
the
feature/transformers-v3
branch
from
ec0f7b3 to
8f92f83
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Migrate translation to browser Web Worker (@huggingface/transformers v3)
Migrates the translation system from a Node.js Worker Thread using @xenova/transformers to a browser Web Worker using @huggingface/transformers v3 (NLLB-200). The worker now runs
entirely on the browser side, removing the dependency on Electron for translation.
Changes
Bug fixes
Dependencies
What is the purpose of this pull request?