From 89d1b863c2c9d18f142e229b89d5254a93501bab Mon Sep 17 00:00:00 2001 From: joshvanl Date: Sun, 9 Nov 2025 17:33:41 +0000 Subject: [PATCH 1/5] Adds Workflow Purge TTL 20251108-RIS-workflow-state-ttl.md Signed-off-by: joshvanl --- 20251108-RIS-workflow-purge-ttl.md | 160 +++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 20251108-RIS-workflow-purge-ttl.md diff --git a/20251108-RIS-workflow-purge-ttl.md b/20251108-RIS-workflow-purge-ttl.md new file mode 100644 index 0000000..2b82865 --- /dev/null +++ b/20251108-RIS-workflow-purge-ttl.md @@ -0,0 +1,160 @@ +# Workflow: History Purge TTL on Completion + +* Author(s): @joshvanl + +## Overview + +This proposal details new functionality to the workflow runtime to give users the ability to delete completed workflow state from the actor state store after some configured time. +All workflow instances may be configured with a unique TTL at workflow execution time. +The default remains that workflow state will _not_ be deleted form the actor state store, and will remain there indefinitely. + +## Background + +It is currently the case that in order for users to delete old workflow state from the actor state store database, they either need to use the Purge Workflow API, or delete state from the database directy, either via out of Dapr database operations, or via using some kind of first class TTL feature of that database. +Users typically want to delete old workflow state after some period of time from when the workflow has reached a terminal state. + +https://github.com/dapr/dapr/issues/9020 + +## Design + +When scheduling a workflow, users will be able to configure some duration which upon elapsing after the workflow has reached a terminal state, the workflow will be purged from the actor state store. +The duration will only start once the workflow has reached either a TERMINATED, COMPLETED, or FAILED state. + +Any duration may be given, i.e. days, weeks, or years. +A duration of `0` may also be given, if the workflow actor state is wished to be deleted immediately after reaching a terminal state. + +### Usage + +#### CLI + +Users can give a Go style duration string when running a workflow from the CLI. + +```bash +$ dapr run my-workflow --purge-ttl=5d +``` + +```bash +$ dapr run my-workflow --purge-ttl=0s +``` + +The new purge reminders will be displayed by: + +```bash +$ dapr scheduler list +NAME BEGIN COUNT LAST TRIGGER +purge-workflow/my-workflow 96h 0 +``` + +#### Go + +```go +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithPurgeTTL(time.Hour*24*5)) +``` + +```go +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithPurgeTTL(0)) +``` + +#### Python + +```python +wfClient.schedule_new_workflow(workflow=my_workflow, putge_ttl=timedelta(days=5)) +``` + +```python +wfClient.schedule_new_workflow(workflow=my_workflow, putge_ttl=timedelta(seconds=0)) +``` + +#### Javascript + +```js +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, putge_ttl: Temporal.Duration.from({days: 5})}) +``` + +```js +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, putge_ttl: Temporal.Duration.from({})}) +``` + +#### .NET + +```dotnet +workflowClient.ScheduleNewWorkflowAsync( + name: nameof(MyWorkflow), + stateTTL: TimeSpan.FromDays(5); +); +``` + +```dotnet +workflowClient.ScheduleNewWorkflowAsync( + name: nameof(MyWorkflow), + stateTTL: TimeSpan.FromSeconds(0) +); +``` + +#### Java + +```java +workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, TODO: @joshvanl); +``` + +```java +workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, TODO: @joshvanl); +``` + +### Runtime + +#### protos + +The following protos will be updated with the new state TTL duration field so it is piped from workflow creation to execution. + +The new option will be added to `CreateInstanceRequest`, populated by the client. + +```proto +message CreateInstanceRequest { + string instanceId = 1; + string name = 2; + // OTHERS + google.protobuf.Duration purgeTTL = 10; // NEW +} +``` + +`ExecutionStartedEvent` will contain the TTL duration which signals the duration after which the workflow has completed should be purged. +This field will be persistent in the history log. +This field will be populated by the durabletask backend executor, piping the field from `CreateInstanceRequest`. + +```proto +message ExecutionStartedEvent { + string name = 1; + // OTHERS + google.protobuf.Duration purgeTTL = 10; // NEW +} +``` + +#### Actors + +Upon workflow reaching a terminal state, after the orchestraion actor has written the result to the actor state store, it will also create an actor reminder if the `purgeTTL` field is present in the execution started event. + +This reminder will target a new actor workflow type, with the reminder name being the instance ID of the workflow. + +The new actor type will follow convention and have the following form: + +``` +dapr.internal...purge-workflow +``` + +Upon activation of the reminder, the new purge actor will be activate, call the purge API on the workflow orchestrator actor for the given instance ID, and then deactivate itself. +Along with the other workflow actor types, this type will be registered on workflow client connection, and unregistration on workflow client disconnection. + +By using a new actor type, this feature is fully backwards compatible since older clients will not register for this new purge workflow type. + + +``` +WORKFLOW COMPLETE -> orestrator -> create purge reminder -...> execute purge reminder -> execute purge actor -> execute purge on orchestrator +``` + +# Alternatives + +Another option is to use the actor TTL state store functionality to delete store keys based on individual key TTls. +This is not appropriate as it _must_ be the case that workflow data be only delete from the state store once the workflow has reached a terminal state. +Not doing so would corrupt the workflow processing. +It is therefore necessary that the Purge API is used to delete the stored data, which itself processes the request inside the same workflow state machine. From 9b0f015ccaea54b4f95adf810d53fc0257bb2460 Mon Sep 17 00:00:00 2001 From: joshvanl Date: Mon, 10 Nov 2025 12:29:22 +0000 Subject: [PATCH 2/5] Fixes & Java example Signed-off-by: joshvanl --- 20251108-RIS-workflow-purge-ttl.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/20251108-RIS-workflow-purge-ttl.md b/20251108-RIS-workflow-purge-ttl.md index 2b82865..f509384 100644 --- a/20251108-RIS-workflow-purge-ttl.md +++ b/20251108-RIS-workflow-purge-ttl.md @@ -6,11 +6,11 @@ This proposal details new functionality to the workflow runtime to give users the ability to delete completed workflow state from the actor state store after some configured time. All workflow instances may be configured with a unique TTL at workflow execution time. -The default remains that workflow state will _not_ be deleted form the actor state store, and will remain there indefinitely. +The default remains that workflow state will _not_ be deleted from the actor state store, and will remain there indefinitely. ## Background -It is currently the case that in order for users to delete old workflow state from the actor state store database, they either need to use the Purge Workflow API, or delete state from the database directy, either via out of Dapr database operations, or via using some kind of first class TTL feature of that database. +It is currently the case that in order for users to delete old workflow state from the actor state store database, they either need to use the Purge Workflow API, or delete state from the database directly, either via out of Dapr database operations, or via using some kind of first class TTL feature of that database. Users typically want to delete old workflow state after some period of time from when the workflow has reached a terminal state. https://github.com/dapr/dapr/issues/9020 @@ -37,7 +37,7 @@ $ dapr run my-workflow --purge-ttl=5d $ dapr run my-workflow --purge-ttl=0s ``` -The new purge reminders will be displayed by: +The new purge reminders will be displayed like: ```bash $ dapr scheduler list @@ -94,18 +94,20 @@ workflowClient.ScheduleNewWorkflowAsync( #### Java ```java -workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, TODO: @joshvanl); +opts.setPurgeTTL(Duration.ofDays(5)); +workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, opts); ``` ```java -workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, TODO: @joshvanl); +opts.setPurgeTTL(Duration.ofSeconds(5)); +workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, opts); ``` ### Runtime #### protos -The following protos will be updated with the new state TTL duration field so it is piped from workflow creation to execution. +The following protos will be updated with the new purge TTL duration field so it is piped from workflow creation to execution. The new option will be added to `CreateInstanceRequest`, populated by the client. @@ -113,7 +115,7 @@ The new option will be added to `CreateInstanceRequest`, populated by the client message CreateInstanceRequest { string instanceId = 1; string name = 2; - // OTHERS + // EXISTING google.protobuf.Duration purgeTTL = 10; // NEW } ``` @@ -125,14 +127,14 @@ This field will be populated by the durabletask backend executor, piping the fie ```proto message ExecutionStartedEvent { string name = 1; - // OTHERS + // EXISTING google.protobuf.Duration purgeTTL = 10; // NEW } ``` #### Actors -Upon workflow reaching a terminal state, after the orchestraion actor has written the result to the actor state store, it will also create an actor reminder if the `purgeTTL` field is present in the execution started event. +Upon workflow reaching a terminal state, after the orchestraion actor has written the result to the actor state store, it will then create an actor reminder if the `purgeTTL` field is present in the execution started event. This reminder will target a new actor workflow type, with the reminder name being the instance ID of the workflow. @@ -142,10 +144,10 @@ The new actor type will follow convention and have the following form: dapr.internal...purge-workflow ``` -Upon activation of the reminder, the new purge actor will be activate, call the purge API on the workflow orchestrator actor for the given instance ID, and then deactivate itself. -Along with the other workflow actor types, this type will be registered on workflow client connection, and unregistration on workflow client disconnection. +Upon activation of the reminder, the new purge actor will be activated, call the purge API on the workflow orchestrator actor for the given instance ID, and then deactivate itself. +Along with the other workflow actor types, this type will be registered on workflow client connection, and unregistered on workflow worker client disconnection. -By using a new actor type, this feature is fully backwards compatible since older clients will not register for this new purge workflow type. +By using a new actor type, this feature is fully backwards compatible as older clients will not register for this new purge workflow type. ``` From 845f133e1a6d7225b631dce87daaeecd49f7066e Mon Sep 17 00:00:00 2001 From: Josh van Leeuwen Date: Mon, 10 Nov 2025 18:58:59 +0000 Subject: [PATCH 3/5] Update 20251108-RIS-workflow-purge-ttl.md Co-authored-by: Albert Callarisa Signed-off-by: Josh van Leeuwen --- 20251108-RIS-workflow-purge-ttl.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/20251108-RIS-workflow-purge-ttl.md b/20251108-RIS-workflow-purge-ttl.md index f509384..c489b5f 100644 --- a/20251108-RIS-workflow-purge-ttl.md +++ b/20251108-RIS-workflow-purge-ttl.md @@ -5,7 +5,7 @@ ## Overview This proposal details new functionality to the workflow runtime to give users the ability to delete completed workflow state from the actor state store after some configured time. -All workflow instances may be configured with a unique TTL at workflow execution time. +All workflow instances may be configured with a unique TTL at workflow scheduling time. The default remains that workflow state will _not_ be deleted from the actor state store, and will remain there indefinitely. ## Background From e087d8d383692d03f27afc443db0152a03e696b8 Mon Sep 17 00:00:00 2001 From: joshvanl Date: Wed, 12 Nov 2025 12:34:17 +0000 Subject: [PATCH 4/5] Update doc to be state retention and use dapr config Signed-off-by: joshvanl --- 20251108-RIS-workflow-purge-ttl.md | 141 ++++++++++++++++++++--------- 1 file changed, 96 insertions(+), 45 deletions(-) diff --git a/20251108-RIS-workflow-purge-ttl.md b/20251108-RIS-workflow-purge-ttl.md index c489b5f..084a095 100644 --- a/20251108-RIS-workflow-purge-ttl.md +++ b/20251108-RIS-workflow-purge-ttl.md @@ -1,11 +1,12 @@ -# Workflow: History Purge TTL on Completion +# Workflow: History State Retention * Author(s): @joshvanl ## Overview -This proposal details new functionality to the workflow runtime to give users the ability to delete completed workflow state from the actor state store after some configured time. -All workflow instances may be configured with a unique TTL at workflow scheduling time. +This proposal details new functionality to the workflow runtime to give users the ability to delete old workflow state from the actor state store after some configured time. +An app scoped configuration policy can be given to set a TTL for all workflows in that app. +All workflow instances may be configured with a unique TTL at workflow scheduling time. The default remains that workflow state will _not_ be deleted from the actor state store, and will remain there indefinitely. ## Background @@ -17,62 +18,110 @@ https://github.com/dapr/dapr/issues/9020 ## Design +A configuration spec will be added to the runtime workflow spec to set a retention policy for all workflow created from that app. When scheduling a workflow, users will be able to configure some duration which upon elapsing after the workflow has reached a terminal state, the workflow will be purged from the actor state store. The duration will only start once the workflow has reached either a TERMINATED, COMPLETED, or FAILED state. +If a retention policy is set at both the app level and the workflow level, the workflow level setting will take precedence. + Any duration may be given, i.e. days, weeks, or years. A duration of `0` may also be given, if the workflow actor state is wished to be deleted immediately after reaching a terminal state. ### Usage +#### Configuration + +```go +type WorkflowSpec struct { + // StateRetentionPolicy defines the retention configuration for workflow + // state once a workflow reaches a terminal state. If not set, workflow + // instances will not be automatically purged. + StateRetentionPolicy *WorkflowStateRetentionPolicy `json:"stateRetentionPolicy,omitempty" yaml:"stateRetentionPolicy,omitempty"` +} + +// WorkflowStateRetentionPolicy defines the retention policy of workflow state +// for workflow instances once they reaches a specific or any terminal state. +// If not set, workflow instances will not be automatically purged. If a +// specific and any terminal state are both set, the specific terminal state +// takes precedence. Accepts duration strings, e.g. "72h" or "30m", including +// immediate values "0s". +type WorkflowStateRetentionPolicy struct { + // AnyTerminal is the TTL for purging workflow instances that reach any + // terminal state. + AnyTerminal *time.Duration `json:"anyTerminal,omitempty" yaml:"anyTerminal,omitempty"` + + // Completed is the TTL for purging workflow instances that reach the + // Completed terminal state. + Completed *time.Duration `json:"completed,omitempty" yaml:"completed,omitempty"` + + // Failed is the TTL for purging workflow instances that reach the Failed + // terminal state. + Failed *time.Duration `json:"failed,omitempty" yaml:"failed,omitempty"` + + // Terminated is the TTL for purging workflow instances that reach the + // Terminated terminal state. + Terminated *time.Duration `json:"terminated,omitempty" yaml:"terminated,omitempty"` +} +``` + +```yaml +kind: Configuration +metadata: + name: wfpolicy +spec: + workflow: + stateRetentionPolicy: + anyTerminal: "5s" + completed: "0s" + failed: "999h" + terminated: "999h" +``` + #### CLI Users can give a Go style duration string when running a workflow from the CLI. ```bash -$ dapr run my-workflow --purge-ttl=5d +$ dapr run my-workflow --state-retention=120h ``` ```bash -$ dapr run my-workflow --purge-ttl=0s +$ dapr run my-workflow --state-retention=0s ``` -The new purge reminders will be displayed like: +The new retention reminders will be displayed like: ```bash $ dapr scheduler list -NAME BEGIN COUNT LAST TRIGGER -purge-workflow/my-workflow 96h 0 +NAME BEGIN COUNT LAST TRIGGER +workflow-retention/my-workflow 96h 0 ``` #### Go ```go -wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithPurgeTTL(time.Hour*24*5)) -``` - -```go -wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithPurgeTTL(0)) +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithStateRetention(time.Hour*24*5)) +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithStateRetention(0)) +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithStateRetentionCompleted(0)) +wf.ScheduleWorkflow(ctx, "my-workflow", workflow.WithStateRetentionFailed(time.Hour*24*5)) ``` #### Python ```python -wfClient.schedule_new_workflow(workflow=my_workflow, putge_ttl=timedelta(days=5)) -``` - -```python -wfClient.schedule_new_workflow(workflow=my_workflow, putge_ttl=timedelta(seconds=0)) +wfClient.schedule_new_workflow(workflow=my_workflow, state_retention=timedelta(days=5)) +wfClient.schedule_new_workflow(workflow=my_workflow, state_retention=timedelta(seconds=0)) +wfClient.schedule_new_workflow(workflow=my_workflow, state_retention_completed=timedelta(seconds=0)) +wfClient.schedule_new_workflow(workflow=my_workflow, state_retention_failed=timedelta(days=5)) ``` #### Javascript ```js -workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, putge_ttl: Temporal.Duration.from({days: 5})}) -``` - -```js -workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, putge_ttl: Temporal.Duration.from({})}) +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, state_retention: Temporal.Duration.from({days: 5})}) +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, state_retention: Temporal.Duration.from({})}) +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, state_retention_completed: Temporal.Duration.from({})}) +workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, state_retention_failed: Temporal.Duration.from({days: 5})}) ``` #### .NET @@ -80,26 +129,22 @@ workflowClient.scheduleNewWorkflow({workflow: MyWorkflow, putge_ttl: Temporal.Du ```dotnet workflowClient.ScheduleNewWorkflowAsync( name: nameof(MyWorkflow), - stateTTL: TimeSpan.FromDays(5); + stateRetention: TimeSpan.FromDays(5); ); -``` - -```dotnet workflowClient.ScheduleNewWorkflowAsync( name: nameof(MyWorkflow), - stateTTL: TimeSpan.FromSeconds(0) + stateRetention: TimeSpan.FromSeconds(0) + stateRetentionCompleted: TimeSpan.FromSeconds(0) + stateRetentionFailed: TimeSpan.FromDays(5) ); ``` #### Java ```java -opts.setPurgeTTL(Duration.ofDays(5)); -workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, opts); -``` - -```java -opts.setPurgeTTL(Duration.ofSeconds(5)); +opts.setStateRetention(Duration.ofDays(5)); +opts.setStateRetentionCompleted(Duration.ofSeconds(0)); +opts.setStateRetentionFailed(Duration.ofDays(5)); workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, opts); ``` @@ -107,7 +152,7 @@ workflowClient.scheduleNewWorkflow(OrderProcessingWorkflow.class, opts); #### protos -The following protos will be updated with the new purge TTL duration field so it is piped from workflow creation to execution. +The following protos will be updated with the new retention policy message so it is piped from workflow creation to execution. The new option will be added to `CreateInstanceRequest`, populated by the client. @@ -115,12 +160,19 @@ The new option will be added to `CreateInstanceRequest`, populated by the client message CreateInstanceRequest { string instanceId = 1; string name = 2; - // EXISTING - google.protobuf.Duration purgeTTL = 10; // NEW + // ... + optional InstanceStateRetentionPolicy retentionPolicy = 10; // NEW +} + +message InstanceStateRetentionPolicy { + optional google.protobuf.Duration allTerminal = 1; + optional google.protobuf.Duration completed = 2; + optional google.protobuf.Duration failed = 3; + optional google.protobuf.Duration terminated = 4; } ``` -`ExecutionStartedEvent` will contain the TTL duration which signals the duration after which the workflow has completed should be purged. +`ExecutionStartedEvent` will contain the retention policy which signals the duration after which the workflow has completed should be purged. This field will be persistent in the history log. This field will be populated by the durabletask backend executor, piping the field from `CreateInstanceRequest`. @@ -128,30 +180,29 @@ This field will be populated by the durabletask backend executor, piping the fie message ExecutionStartedEvent { string name = 1; // EXISTING - google.protobuf.Duration purgeTTL = 10; // NEW + optional InstanceStateRetentionPolicy = 10; // NEW } ``` #### Actors -Upon workflow reaching a terminal state, after the orchestraion actor has written the result to the actor state store, it will then create an actor reminder if the `purgeTTL` field is present in the execution started event. +Upon workflow reaching a terminal state, after the orchestration actor has written the result to the actor state store, it will then create an actor reminder if the state retention policy field is present in the execution started event or as the app ID workflow configuration. -This reminder will target a new actor workflow type, with the reminder name being the instance ID of the workflow. +This reminder will target a new actor workflow type, with the actor ID being the instance ID of the workflow. The new actor type will follow convention and have the following form: ``` -dapr.internal...purge-workflow +dapr.internal...retentioner ``` -Upon activation of the reminder, the new purge actor will be activated, call the purge API on the workflow orchestrator actor for the given instance ID, and then deactivate itself. +Upon activation of the reminder, the new retentioner actor will be activated, call the purge API on the workflow orchestrator actor for the given instance ID, and then deactivate itself. Along with the other workflow actor types, this type will be registered on workflow client connection, and unregistered on workflow worker client disconnection. By using a new actor type, this feature is fully backwards compatible as older clients will not register for this new purge workflow type. - ``` -WORKFLOW COMPLETE -> orestrator -> create purge reminder -...> execute purge reminder -> execute purge actor -> execute purge on orchestrator +WORKFLOW COMPLETE -> orestrator -> create retentioner reminder -...> execute retentioner reminder -> execute retentioner actor -> execute purge on orchestrator ``` # Alternatives From c75f03da5c54417ff370f9d5da47edd03f4f8df6 Mon Sep 17 00:00:00 2001 From: joshvanl Date: Wed, 12 Nov 2025 12:37:53 +0000 Subject: [PATCH 5/5] Show how to delete the workflow retention from the CLI Signed-off-by: joshvanl --- 20251108-RIS-workflow-purge-ttl.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/20251108-RIS-workflow-purge-ttl.md b/20251108-RIS-workflow-purge-ttl.md index 084a095..7efea8b 100644 --- a/20251108-RIS-workflow-purge-ttl.md +++ b/20251108-RIS-workflow-purge-ttl.md @@ -97,6 +97,13 @@ NAME BEGIN COUNT LAST TRIGGER workflow-retention/my-workflow 96h 0 ``` +A user is able to delete a retention reminder by either manually purging the workflow or deleting the reminder directly. + +```bash +$ dapr workflow purge my-workflow +$ dapr scheduler delete workflow-retention/my-workflow +``` + #### Go ```go