Skip to content

Unsupported Type: SparkConfigurations #32

@jreitz43

Description

@jreitz43

Deployment throws error "ASWT0029: Unknown object type: SparkConfiguration" when deploying a notebook (example shown below) that has a reference to a custom spark configuration. I believe this may be a very similar issue to #11 where SparkConfiguration would need to be added to the allowed valid types in private/!SynapseObject.class.ps1 as well as private/Get-SynapseObjectByName.ps1 so that it can pretend that the object referenced is valid and exists.
Notebook:

{
	"name": "nb_example",
	"properties": {
		"folder": {
			"name": "utility"
		},
		"nbformat": 4,
		"nbformat_minor": 2,
		"bigDataPool": {
			"referenceName": "sparkSm",
			"type": "BigDataPoolReference"
		},
		"targetSparkConfiguration": {
			"referenceName": "CustomSparkConfig",
			"type": "SparkConfigurationReference"
		},

Failure:

**mVERBOSE: A***lyzing notebook dependencies...
**mVERBOSE: Folder: D:\a\***\drop\***\notebook
**mVERBOSE: - nb_example.json
Failure occurred while publishing artifacts to ***
##[debug]Error record:
##[debug]Exception: D:\a\_temp\a560ab36-63de-480f-888a-bc877***f23f0a.ps***:38
##[debug]     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##[debug]     | ASWT0029: Unknown object type: SparkConfiguration.

azure.synapse.tools Version 0.23.000

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions