Skip to content

Allow using video for ControlNet for target type image#1362

Merged
mcmonkey4eva merged 5 commits intomcmonkeyprojects:masterfrom
jtreminio:controlnet-video-support-2
May 2, 2026
Merged

Allow using video for ControlNet for target type image#1362
mcmonkey4eva merged 5 commits intomcmonkeyprojects:masterfrom
jtreminio:controlnet-video-support-2

Conversation

@jtreminio
Copy link
Copy Markdown
Contributor

Note that "Preview" might still not work here.

Hitting the button results in:

[vost#0:0/libx264 @ 0x15d05fc0] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x15cfa340] Error sending frames to consumers: Generic error in an external library
[vf#0:0 @ 0x15cfa340] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 0x15cfa340] Terminating thread with return code -542398533 (Generic error in an external library)
[vost#0:0/libx264 @ 0x15d05fc0] Could not open encoder before EOF
[vost#0:0/libx264 @ 0x15d05fc0] Task finished with error code: -22 (Invalid argument)
[vost#0:0/libx264 @ 0x15d05fc0] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0x15d05640] Nothing was written into output file, because at least one of its streams received no packets.

I worked around this by following your example here.

CleanShot 2026-05-01 at 18 43 43

@jtreminio
Copy link
Copy Markdown
Contributor Author

Second commit fixes issue with "Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height":

CleanShot.2026-05-01.at.19.23.33.mp4

if (g.UserInput.Get(T2IParamTypes.ControlNetPreviewOnly))
{
g.CurrentMedia = imageNodeActual.WithPath(preprocActual);
if (g.CurrentMedia.DataType == WGNodeData.DT_VIDEO)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this inside the ControlNetPreviewOnly area?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved it out to apply to all preprocessors. As mentioned in discord, the incorrect resolution seems to happen for all preprocessors I tested that actually finished:

00:30:06.196 [Info] User local requested 1 image with model 'z-image/Sophos_ZiT_-_v1-1.safetensors'...
00:30:38.745 [Warning] [ComfyUI-1/STDERR] ffmpeg error: [libx264 @ 0x1937f500] height not divisible by 2 (1024x1497)
00:30:38.745 [Warning] [ComfyUI-1/STDERR] [vost#0:0/libx264 @ 0x1937efc0] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
00:30:38.749 [Warning] [ComfyUI-1/STDERR] Traceback (most recent call last):
00:30:38.749 [Warning] [ComfyUI-1/STDERR]   File "/workspace/dlbackend/ComfyUI/execution.py", line 534, in execute
00:30:38.749 [Warning] [ComfyUI-1/STDERR]     output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
00:30:38.749 [Warning] [ComfyUI-1/STDERR]                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
00:30:38.749 [Warning] [ComfyUI-1/STDERR]   File "/workspace/dlbackend/ComfyUI/execution.py", line 334, in get_output_data
00:30:38.749 [Warning] [ComfyUI-1/STDERR]     return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
00:30:38.749 [Warning] [ComfyUI-1/STDERR]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
00:30:38.749 [Warning] [ComfyUI-1/STDERR]   File "/workspace/dlbackend/ComfyUI/execution.py", line 308, in _async_map_node_over_list
00:30:38.749 [Warning] [ComfyUI-1/STDERR]     await process_inputs(input_dict, i)
00:30:38.749 [Warning] [ComfyUI-1/STDERR]   File "/workspace/dlbackend/ComfyUI/execution.py", line 296, in process_inputs
00:30:38.749 [Warning] [ComfyUI-1/STDERR]     result = f(**inputs)
00:30:38.749 [Warning] [ComfyUI-1/STDERR]              ^^^^^^^^^^^
00:30:38.749 [Warning] [ComfyUI-1/STDERR]   File "/workspace/src/BuiltinExtensions/ComfyUIBackend/ExtraNodes/SwarmComfyExtra/SwarmSaveAnimationWS.py", line 144, in save_images
00:30:38.749 [Warning] [ComfyUI-1/STDERR]     raise Exception(f"ffmpeg failed: {f_err}")
00:30:38.749 [Warning] [ComfyUI-1/STDERR] Exception: ffmpeg failed: [libx264 @ 0x1937f500] height not divisible by 2 (1024x1497)

@mcmonkey4eva mcmonkey4eva merged commit b4f30a9 into mcmonkeyprojects:master May 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants