TongFlow : An Open-Source Multi-Modal GenAI Workflow Studio

190c80bcbf0c2978f5a316504e51a30f.mp4

English · 简体中文 · 日本語

Demo Examples

Workflow	Result
Basic — Type text (Add), generate images (Transform), then blend them into one (Compose).
Intermediate — (Add topic → write script → generate speech) + (character description → generate image) → lip-synced video = talking-head avatar.	digitalhuman.mp4
Advanced — Generate lyrics + song + characters + scenes + storyboard → produce a music video.	mv.mp4

With TongFlow, you can expand your imagination and stretch your ideas with generative AI, just have a try now!

How To Start

The TongFlow desktop app is a lightweight (~10 MB) shell around the cloud studio at app.tongflow.com — install it, sign in, and start creating. The cloud studio also runs in any modern browser.

Step 1 — Install the desktop app

Download the installer for your platform, install it, and open it.

• macOS (Universal — Apple Silicon & Intel): TongFlow-mac-universal.dmg
• Windows: TongFlow-win-x64.msi

All builds are on the Releases page.

macOS: the builds are not yet notarized with Apple, so Gatekeeper will block the first launch ("TongFlow is damaged and can't be opened"). After moving the app to Applications, clear the quarantine flag once and it opens normally:

xattr -cr /Applications/TongFlow.app

Download from this page directly — installers passed through chat apps (e.g. WeChat) may be renamed or re-flagged.

Step 2 — Sign in and create

Sign in with Google or WeChat and start creating — the cloud studio manages plugins and execution for you.

Prefer a fully local, account-free TongFlow? That's what self-hosting is for — see Run from source or Run with Docker, then follow Self-host setup. (The desktop app up to v0.1.13 bundled this local runtime; those installers remain on the Releases page.)

Core Concept

• All models: AI models can be thought of as a modality transform (e.g. LLMs are text→text, image models are text→image, speech models are text→audio, and so on). TongFlow wraps each capability as a node.

• All modalities: TongFlow supports almost every modality and file format that people actually ship over the web.

• Low barrier, high ceiling: no complex AI parameters to learn, no manual node connecting; just three operations — add, transform, and combine — to arrange ideas freely. And by orchestrating AI models freely, you can generate unique creations and works of your own.

• Open ecosystem: TongFlow's plugin-based design lets every platform package its own independent plugins, and we provide at least one official implementation plugin for each capability node. The core stays small, the ecosystem stays open.

What’s Defined

✅ = available out of the box with an official plugin · ⬜ = node exists in the canvas but has no official plugin yet (planned).

Add

• ✅ Text input: type text and add a text node.
• ✅ Add image: pick a local file and add an image node.
• ✅ Add photo: capture with the device camera and add an image node.
• ✅ Add sketch: draw on the canvas and add an image node.
• ✅ Add audio: pick a local audio file and add an audio node.
• ✅ Record audio: record with the mic and add an audio node.
• ✅ Add video: pick a local video file and add a video node.
• ✅ Record video: record with the camera and add a video node.
• ✅ Add document: pick a local file and add a document node.
• ✅ Add URL: fetch a page from a link and add text, image, audio, or video nodes.
• ✅ Add 3D model: choose a local model file and add a 3D model node.

Transform

Text

• ✅ Generate / rewrite: create or edit copy from a prompt.

Image

• ✅ Image generation: images from text.
• ✅ Image editing: inpaint, edit, or redraw with instructions.
• ✅ Image understanding: captions, Q&A, or descriptions from an image.
• ✅ Image upscaling: enlarge for sharper detail.

Video

• ✅ Video generation: video from text.
• ✅ Image-to-video: animate a still into motion.
• ✅ First/last-frame video: two key images to interpolate a clip.
• ✅ Images → video: multi-image reference fusion — several reference images plus text into a new video.
• ✅ Video understanding: summaries or descriptions from video.
• ✅ Video upscaling: higher-resolution output.
• ✅ Extract first / last frame: grab a frame as an image.
• ✅ Video editing: edit a video from a text instruction.
• ✅ Subtitle removal: clean subtitles from a video.
• ✅ Watermark removal: remove watermarks from a video.

Audio

• ✅ Music generation: music from text, with optional reference-audio conditioning.
• ✅ Audio understanding: describe a clip (music, speech, or ambient sound) in text.
• ✅ Music repaint: regenerate a chosen time range of a song.
• ✅ Music cover: restyle a song via a caption and/or a reference track.
• ✅ Add track / complete arrangement: generate one new stem over a mix, or fill in missing tracks.
• ✅ Music brief: one-sentence idea → lyrics, style tags, BPM, key, and duration.
• ✅ Speech synthesis: text-to-speech — preset style, voice clone (reference audio), or instruction-driven.
• ✅ Speech recognition: transcribe speech from audio or video.
• ✅ Noise reduction: denoise audio.
• ⬜ Speaker diarization: separate audio by speaker.
• ⬜ Voice / timbre replacement: replace or clone a voice with a reference sample.
• ✅ Multi-track / vocal-accompaniment separation: isolate vocals, drums, bass, guitar, and 8 more stems.

Combine

• ✅ Image fusion: blend or edit multiple references into one image.
• ✅ Lip sync: audio + video → video (lip-sync); also audio + image → video and audio + text → video variants.
• ✅ Character swap: video + reference (scene blend / character replacement), Animate Mix-style generation.
• ✅ Motion transfer: video + reference (motion / retarget), Animate Move-style generation.
• ✅ Combine text: merge multiple text nodes into one.

Other

• ✅ Image → 3D: single-view 3D model from an image.
• ✅ Document → text: extract plain text from documents.
• ✅ Link → text: turn page content into text.

Helpers

• ✅ Concatenate clips: join multiple videos end to end.
• ✅ Mux audio + video: merge into one file.
• ✅ Split by shots: cut a long video into segments by scene.
• ✅ Split video & audio: demux a video into separate video and audio tracks.
• ✅ Extract audio track: pull audio into its own asset.
• ✅ Split long text: break a long passage into chunks.
• ✅ Merge / tidy text blocks: combine segments (use the auto-merge option).
• ✅ Filter or drop clips: drop unwanted clips by rule or selection.
• ✅ Arrange & batch groups: group and arrange text/clip batches for downstream processing.

Official plugins

The official GPU/CPU plugins currently run on Modal — up to $30/month of free GPU compute (H100/A100, etc.). See Self-host setup for the MODAL_TOKEN_* setup. Any other platform can publish its own plugins the same way.

API plugins

• tongflow-api-openrouter-free — default gen_text route via OpenRouter's free models
• tongflow-api-gemini — Google Gemini for gen_text and image generation / editing / fusion (Nano Banana)
• tongflow-api-openai — OpenAI for gen_text and image generation / editing / fusion (gpt-image-2)
• tongflow-api-bytedance — Volcengine Ark (Doubao Seedance 2.0) for text/image/audio → video
• tongflow-api-apimart — APIMart gateway with a per-node model picker: image gen/edit (Z-Image, Seedream, Nano Banana, GPT-Image), text/image → video (Kling, VEO3, Sora2, Seedance), gen_text (GPT-5, Claude, Gemini), Whisper transcription and TTS
• tongflow-api-agnes — Agnes AI gateway: gen_text / text tools / image understanding (agnes-2.0-flash), image generation / editing / fusion (agnes-image-2.x-flash), and text / image / first-last-frame → video (agnes-video-v2.0)

GPU/CPU plugins

• tongflow-modal-ffmpeg — transcoding, muxing, media pipelines
• tongflow-modal-pyscenedetect — shot-boundary detection for splitting clips
• tongflow-modal-z-image — Z-Image text-to-image
• tongflow-modal-ernie-image — ERNIE Image text-to-image (alternative)
• tongflow-modal-flux2-klein9b — FLUX.2 Klein 9B multi-reference fusion / image editing
• tongflow-modal-boogu — Boogu-Image-0.1 (fp8) text-to-image (dense bilingual text) & single-reference image editing
• tongflow-modal-ltx — LTX-2.3 text / image-to-video
• tongflow-modal-fastwan — FastWan-QAD-FP8 fast text-to-video (3-step distilled Wan2.1-1.3B)
• tongflow-modal-infinitetalk — InfiniteTalk audio-driven lip-sync (audio + image / video → talking-head video)
• tongflow-modal-wan-animate — Wan-Animate character swap & motion transfer (video + reference)
• tongflow-modal-scail2 — SCAIL-2 controlled character animation (image + driving video; same two slots as wan-animate)
• tongflow-modal-bernini — Bernini-R 1.3B unified video renderer (text/image → image/video, video editing, subtitle / watermark removal)
• tongflow-modal-sam3 — SAM 3 / SAM 3.1 text-guided matting: cut every instance of a described concept out of an image (transparent PNG) or track it through a video (green-screen matte)
• tongflow-modal-triposplat — TripoSplat single image → 3D Gaussian splat
• tongflow-modal-sam-3d-objects — SAM 3D Objects single image → 3D Gaussian splat of the foreground object (auto mask, robust to occlusion/clutter; alternative)
• tongflow-modal-sam-3d-body — SAM 3D Body single image → full-body 3D human mesh GLB (multi-person, MHR rig; alternative)
• tongflow-modal-seedvr2 — SeedVR2 image / video super-resolution
• tongflow-modal-gemma4 — Gemma-4 multimodal text (image / video understanding)
• tongflow-modal-qwen3asr — Qwen3 speech recognition
• tongflow-modal-qwen3tts — Qwen3 text-to-speech
• tongflow-modal-whisper — Whisper speech recognition with timestamps (alternative)
• tongflow-modal-ace-step — ACE-Step 1.5 music suite: text-to-music (sft / base / turbo selectable), repaint, cover, stem extraction, add-track, arrangement completion, music brief, and music understanding
• tongflow-modal-levo — LeVo 2 / SongGeneration text-to-music (multilingual, commercial-grade)
• tongflow-modal-sam-audio — SAM-Audio text-prompted sound separation: noise reduction, vocal isolation, and free-text stem extraction ("the piano in the background")
• tongflow-modal-docling — Docling document → text
• tongflow-modal-paddle — PaddleOCR document → text
• tongflow-modal-unlimited-ocr — Unlimited-OCR long-horizon document / PDF → text
• tongflow-modal-crawl4ai — Crawl4AI URL / link → text
• tongflow-modal-scrapling — Scrapling stealth-browser URL / link → text

Run from source

pnpm install
pnpm plugins:install   # clone official plugins into plugins/
pnpm start:prod        # builds once, then serves at http://localhost:3000

Requires Node (with pnpm) and a Python 3.10+ interpreter on your PATH (set PYTHON to point at a specific one). Plugins run as local Python processes; TongFlow provisions an isolated venv for them automatically and installs each plugin's requirements.txt on first use — no manual Python setup.

Open http://localhost:3000 and the canvas is live. Then follow Self-host setup (credentials go in the in-app Settings dialog, or a project .env).

Run with Docker

A self-host image is published to GHCR — no Node/Python/pnpm setup required:

docker run -d -p 3000:3000 \
  -v tongflow-data:/data -v tongflow-plugins:/plugins \
  ghcr.io/tong-io/tongflow:latest

Then open http://localhost:3000. Or with Compose (clones this repo's docker-compose.yml):

docker compose up -d

To build the image yourself instead of pulling: docker build -t tongflow .

Data & credentials. Everything writable lives in the /data volume (SQLite db, uploads, settings). API keys are optional — set them in the in-app Settings dialog, or pass them at launch (-e OPENROUTER_API_KEY=…); supported keys: OPENROUTER_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, MODAL_TOKEN_ID / MODAL_TOKEN_SECRET.

Plugins. The image ships no plugins — install them from the in-app plugin manager (first install needs network access to GitHub). On first run, a plugin provisions a shared Python venv under /data/.tongflow/plugin-venv (installs the SDK + the plugin's requirements.txt from PyPI), so the first run is slower and needs network. Modal-backed plugins additionally need a Modal token.

Self-host setup (plugins & credentials)

A self-hosted TongFlow ships with no plugins pre-installed, and the canvas is preloaded with an example workflow. Three steps get it running:

1 — Install plugins

Open the plugin manager (the blocks icon, top-right) and install what you need. Newly installed plugins are usable immediately, no restart.

To run the preloaded example workflow (text → image → fusion → video), install these three plugins:

• tongflow-modal-z-image — text-to-image
• tongflow-modal-flux2-klein9b — image fusion / blending
• tongflow-modal-ltx — image-to-video

These run on Modal (up to $30/month of free GPU compute). Add MODAL_TOKEN_ID / MODAL_TOKEN_SECRET in Settings; create a token at modal.com/settings/tokens. Any other platform can publish its own plugins the same way.

Browse the full catalog — the official API plugins (OpenAI / Gemini / OpenRouter) and other GPU/CPU plugins — in the plugin manager.

2 — Configure credentials

Open Settings (the gear icon, top-right) and add the environment variables your plugins need — e.g. OPENAI_API_KEY for the API plugins, or the credentials your GPU/CPU plugins require.

Plugin credentials live in Settings. TongFlow is platform-agnostic and hardcodes no provider: the Settings dialog is a generic key/value editor for environment variables passed to plugins. Each plugin's README documents the keys it needs. Values are stored locally and take effect without a restart.

3 — Run the example workflow

Run the preloaded example node by node, or switch to Execute Mode and hit the run button to run the whole thing in one click.

Custom plugins

Every runnable node is backed by a contract — the ABI (config/tongflow.abi.json) — that defines what capabilities exist and what each one's input/output looks like, independent of who implements it. A plugin is just a small Python package that picks one or more ABI slots and supplies the how, annotated against the ABI-generated types via the tongflow Python SDK.

The full development flow — the ABI, the @node_slot decorator, the SDK, directory layout, and how to publish — lives in docs/plugins.md.

Community

Join the community on Discord or scan the WeChat group QR code below.

Business

For business inquiries, please contact business@tongflow.com.

• Open-source model owners: I can integrate your models so users can try them out smoothly.
• Enterprise: I can help you deploy on your local GPU, build custom nodes and plugins, and more.
• Platform / router: I can integrate your APIs.
• VCs: I’m interested in partnering on tongflow.com, a cloud-hosted AI studio.

Open-Source

If you like this project, a Star on GitHub helps a lot. Thank you.

License

TongFlow is dual-licensed:

• AGPL-3.0 — free for individuals, research, open-source projects, and anyone willing to comply with the AGPL (including its Section 13 network/source-disclosure obligation).
• Commercial License — for organizations that want to use TongFlow in closed-source or SaaS products without AGPL's source-disclosure obligation, or that need warranties and platform support. Contact business@tongflow.com.

This covers the entire repository, including the sdk/ directory (the tongflow PyPI package). Contributions are covered by our CLA.

Star History

Star History