Skip to content

fix: Asynchronous Pipeline Support and explicit use of utf-8 encoding when reading module files#594

Open
lizhenisu wants to merge 2 commits into
open-webui:mainfrom
lizhenisu:main
Open

fix: Asynchronous Pipeline Support and explicit use of utf-8 encoding when reading module files#594
lizhenisu wants to merge 2 commits into
open-webui:mainfrom
lizhenisu:main

Conversation

@lizhenisu
Copy link
Copy Markdown

The Issue

Fix: Force UTF-8 encoding when loading pipeline modules to resolve UnicodeDecodeError on Windows

On Windows, the default encoding for open() is often system-dependent (e.g., GBK or CP1252). If a user writes a custom pipe function containing UTF-8 characters (like Chinese prompts), the loader fails to read the file, raising a UnicodeDecodeError.

Error Log:

UnicodeDecodeError: 'gbk' codec can't decode byte 0x97 in position ...: illegal multibyte sequence

Reproduction Steps

  1. On a Windows machine.
  2. Create a custom pipeline file with the following content (containing Chinese characters):
# ... inside the pipe function
ollama_messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "把图片内容转成文字"}, # <--- This triggers the error
            image_part
        ]
    }
]
# ...

@lizhenisu lizhenisu changed the title fix: explicit use of utf-8 encoding when reading module files fix: explicit use of utf-8 encoding when reading module files and Asynchronous Pipeline Support Mar 19, 2026
@lizhenisu lizhenisu changed the title fix: explicit use of utf-8 encoding when reading module files and Asynchronous Pipeline Support fix: Asynchronous Pipeline Support and explicit use of utf-8 encoding when reading module files Mar 19, 2026
@WhatIsData
Copy link
Copy Markdown

WhatIsData commented Mar 19, 2026

@tjbck Would you be able to handle the pr for giving support to allow the pipe within Pipeline to be async?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants