Bugfix conversion of single activities in pipeline json files#49
Open
Alex-Swann wants to merge 10 commits intoAzure-Player:mainfrom
Open
Bugfix conversion of single activities in pipeline json files#49Alex-Swann wants to merge 10 commits intoAzure-Player:mainfrom
Alex-Swann wants to merge 10 commits intoAzure-Player:mainfrom
Conversation
…ndling dynamic variables
Member
|
Thank you, Alex for your contribution. Could you please add a unit test to cover the use case? |
Author
I hadn't spotted those. Thanks, I spotted an edge case too. Let me know if they align with both the fix and everything else the PS file is responsible for 👍 |
…String token handling instead of unreliable command version check
…equestHeader token handling
… instead of BSTR, detect by type not version
…ity pipeline object deserialization
…k, verify via side-effects
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bugfix conversion of single activities in pipeline json files when handling dynamic variables.
If a pipeline has a variable that needs to be dynamic to accommodate this being utilised in different deployment environments (DEV, UAT, PROD), e.g.
{ "name": "Run all notebooks", "properties": { "activities": [ { "name": "ForEach Notebook", "type": "ForEach", "dependsOn": [], "userProperties": [], "typeProperties": { "items": { "value": "<an_array_of_string_values_exposed_via_@item()>", "type": "Expression" }, "activities": [ { "name": "Child - Pipeline", "type": "ExecutePipeline", "dependsOn": [], "policy": { "secureInput": false }, "userProperties": [], "typeProperties": { "pipeline": { "referenceName": "Child - Pipeline", "type": "PipelineReference" }, "waitOnCompletion": true, "parameters": { "StorageUrl": { "value": "@variables('StorageUrl')", "type": "Expression" } } } } ] } } ], "variables": { "StorageUrl": { "type": "String", "defaultValue": "test-url" } }, "folder": { "name": "Core Code" }, "annotations": [] } }the serialisation and deserialisation process in
Save-SynapseObjectAsFile.ps1means the activities array, and any nested activities, are accidentally converted back into a object as opposed to an array of objects.{ "name": "Run all notebooks", "properties": { "activities": { "name": "ForEach Notebook", "type": "ForEach", "dependsOn": [], "userProperties": [], "typeProperties": { "items": { "value": "<an_array_of_string_values_exposed_via_@item()>", "type": "Expression" }, "activities": { "name": "Child - Pipeline", "type": "ExecutePipeline", "dependsOn": [], "policy": { "secureInput": false }, "userProperties": [], "typeProperties": { "pipeline": { "referenceName": "Child - Pipeline", "type": "PipelineReference" }, "waitOnCompletion": true, "parameters": { "StorageUrl": { "value": "@variables('StorageUrl')", "type": "Expression" } } } } } }, "variables": { "StorageUrl": { "type": "String", "defaultValue": "test-url" } }, "folder": { "name": "Core Code" }, "annotations": [] } }This fails Synapse's REST-API validations when attempting to deploy. This PR is to start the conversation around addressing it (or fix it altogether).