Skip to content

Bugfix conversion of single activities in pipeline json files#49

Open
Alex-Swann wants to merge 10 commits intoAzure-Player:mainfrom
Alex-Swann:bugfix/pipeline-notebook-deployments
Open

Bugfix conversion of single activities in pipeline json files#49
Alex-Swann wants to merge 10 commits intoAzure-Player:mainfrom
Alex-Swann:bugfix/pipeline-notebook-deployments

Conversation

@Alex-Swann
Copy link
Copy Markdown

@Alex-Swann Alex-Swann commented Mar 27, 2026

Bugfix conversion of single activities in pipeline json files when handling dynamic variables.

If a pipeline has a variable that needs to be dynamic to accommodate this being utilised in different deployment environments (DEV, UAT, PROD), e.g.

{
	"name": "Run all notebooks",
	"properties": {
		"activities": [
			{
				"name": "ForEach Notebook",
				"type": "ForEach",
				"dependsOn": [],
				"userProperties": [],
				"typeProperties": {
					"items": {
						"value": "<an_array_of_string_values_exposed_via_@item()>",
						"type": "Expression"
					},
					"activities": [
						{
							"name": "Child - Pipeline",
							"type": "ExecutePipeline",
							"dependsOn": [],
							"policy": {
								"secureInput": false
							},
							"userProperties": [],
							"typeProperties": {
								"pipeline": {
									"referenceName": "Child - Pipeline",
									"type": "PipelineReference"
								},
								"waitOnCompletion": true,
								"parameters": {
									"StorageUrl": {
										"value": "@variables('StorageUrl')",
										"type": "Expression"
									}
								}
							}
						}
					]
				}
			}
		],
		"variables": {
			"StorageUrl": {
				"type": "String",
				"defaultValue": "test-url"
			}
		},
		"folder": {
			"name": "Core Code"
		},
		"annotations": []
	}
}

the serialisation and deserialisation process in Save-SynapseObjectAsFile.ps1 means the activities array, and any nested activities, are accidentally converted back into a object as opposed to an array of objects.

{
	"name": "Run all notebooks",
	"properties": {
		"activities": {
		          "name": "ForEach Notebook",
		          "type": "ForEach",
		          "dependsOn": [],
		          "userProperties": [],
		          "typeProperties": {
			  "items": {
				  "value": "<an_array_of_string_values_exposed_via_@item()>",
				  "type": "Expression"
			  },
			"activities": {
			        "name": "Child - Pipeline",
			        "type": "ExecutePipeline",
			        "dependsOn": [],
			        "policy": {
				        "secureInput": false
			        },
			        "userProperties": [],
			        "typeProperties": {
				        "pipeline": {
					        "referenceName": "Child - Pipeline",
					        "type": "PipelineReference"
				        },
				        "waitOnCompletion": true,
				        "parameters": {
					"StorageUrl": {
						"value": "@variables('StorageUrl')",
						    "type": "Expression"
					 }
				    }
			       }
		          }
		     }
		},
		"variables": {
			"StorageUrl": {
				"type": "String",
				"defaultValue": "test-url"
			}
		},
		"folder": {
			"name": "Core Code"
		},
		"annotations": []
	}
}

This fails Synapse's REST-API validations when attempting to deploy. This PR is to start the conversation around addressing it (or fix it altogether).

@Alex-Swann Alex-Swann changed the title Bugfix conversion of single activities in pipeline json files when ha… Bugfix conversion of single activities in pipeline json files Mar 27, 2026
@NowinskiK
Copy link
Copy Markdown
Member

Thank you, Alex for your contribution. Could you please add a unit test to cover the use case?

@Alex-Swann
Copy link
Copy Markdown
Author

Thank you, Alex for your contribution. Could you please add a unit test to cover the use case?

I hadn't spotted those. Thanks, I spotted an edge case too. Let me know if they align with both the fix and everything else the PS file is responsible for 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants