Skip to content

Support resuming and forking workflows on their original queue #305

@jonjanisch

Description

@jonjanisch

We should have the ability to resume or fork a workflow on a given queue. For example, imagine we're using a DBOS queue to avoid overloading a fragile service that can only handle 40 req/s. That service goes down and now you have 15K workflows in ERROR state - all failing on the same step.

Ideally, when the service comes back online, we would want to iterate over all the failed workflows, forking them back onto the original queue so they can complete successfully. If we simply forked them naively, we may overwhelm the external service causing further downtime.

Resuming Cancelled Workflows on a Queue

When a workflow is cancelled, currently it's queue_name is set to null. This seems by design. And resumeWorkflow will bypass the queue and run the workflow immediately.

From my perspective, the default behavior should have been that if you cancel a workflow that was on a queue and resume it via resumeWorkflow, it should resume on the same queue it originally belonged to (with an option to run immediately to bypass). However, since changing this default now would break existing clients, I think adding an overloaded method would suffice, e.g.

DBOS.resumeWorkflow(String workflowId, ResumeOptions resumeOptions);

The downside is that the original queueName is lost so the developer would have to know this workflow belonged to queue XYZ. For example, if this was exposed via the dbos console, it would have to prompt for the queue name to resume on (or guess based on recent workflows).

Forking Workflows on a Queue

Similarly, when we fork a workflow, we should also be able to specify the queueName in ForkOptions. Currently, the code is using the DBOS_INTERNAL_QUEUE.

EDIT: Unlike CANCELLED workflows, workflows in the ERROR status still have their original queue_name. So perhaps in ForkOptions it may be sufficient to simply have a config options like useSourceQueue. Alternatively, simply having queueName would allow it to be more flexible and would be more consistent with StartWorkflowOptions.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions