Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/porter-integration-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -194,3 +194,8 @@ jobs:
uses: ./.github/workflows/integ-reuseable-workflow.yml
with:
test_name: state_test
persistent_params_test:
name: Persistent Parameters Integration Test
uses: ./.github/workflows/integ-reuseable-workflow.yml
with:
test_name: persistent_params_test
6 changes: 6 additions & 0 deletions .github/workflows/porter-integration-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,9 @@ jobs:
with:
test_name: state_test
registry: ${{inputs.registry}}
persistent_params_test:
name: Persistent Parameters Integration Test
uses: getporter/porter/.github/workflows/integ-reuseable-workflow.yml@main
with:
test_name: persistent_params_test
registry: ${{inputs.registry}}
68 changes: 68 additions & 0 deletions docs/content/docs/bundle/manifest/file-format/1.2.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: Porter Manifest File Format 1.2.0
description: The 1.2.0 file format for the Porter manifest, porter.yaml
layout: single
---

Bundle manifest file format version 1.2.0 requires the `persistent-parameters` [experimental feature flag].
Comment thread
kichristensen marked this conversation as resolved.

## Changes

Add `persistent` field to parameter definitions.

* Add `parameters.persistent` which, when `true`, automatically wires up a matching output so the
parameter value is remembered across bundle executions (install → upgrade, install → uninstall).

## Example

```yaml
schemaVersion: 1.2.0
name: myapp
version: 1.0.0
registry: localhost:5000

mixins:
- exec

parameters:
- name: resource-group
type: string
persistent: true

install:
- exec:
description: "Deploy to ${bundle.parameters.resource-group}"
command: deploy.sh

upgrade:
- exec:
description: "Upgrade in ${bundle.parameters.resource-group}"
command: upgrade.sh
# resource-group is automatically provided from the install output

uninstall:
- exec:
description: "Remove from ${bundle.parameters.resource-group}"
command: teardown.sh
# resource-group is automatically provided from the install output
```

## Using Persistent Parameters

Enable the feature with the `--experimental` flag:

```console
export PORTER_EXPERIMENTAL=persistent-parameters
porter build
Comment thread
kichristensen marked this conversation as resolved.
```

Or set it in `~/.porter/config.toml`:

```toml
experimental = ["persistent-parameters"]
```

See [Persisting Data Between Bundle Actions] for a full guide.

[experimental feature flag]: /docs/configuration/configuration/#experimental
[Persisting Data Between Bundle Actions]: /docs/development/authoring-a-bundle/persisting-data/
15 changes: 9 additions & 6 deletions docs/content/docs/bundle/manifest/file-format/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ We have a [Porter Visual Studio Code extension][vscode] that provides autocomple

Below are schema versions for the Porter manifest, and the corresponding Porter version that supports it.

| Schema Type | Schema Version | Porter Version |
|-------------|-----------------------------------|------------------|
| Bundle | (none) | v0.38.* |
| Bundle | [1.0.0-alpha.1](./1.0.0-alpha.1/) | v1.0.0-alpha.14+ |
| Bundle | [1.0.0](./1.0.0/) | v1.0.0-beta.2+ |
| Bundle | [1.0.1](./1.0.1/) | v1.0.9+ |
| Schema Type | Schema Version | Porter Version | Experimental Flag |
|-------------|-----------------------------------|------------------|-----------------------------|
| Bundle | (none) | v0.38.* | |
| Bundle | [1.0.0-alpha.1](./1.0.0-alpha.1/) | v1.0.0-alpha.14+ | |
| Bundle | [1.0.0](./1.0.0/) | v1.0.0-beta.2+ | |
| Bundle | [1.0.1](./1.0.1/) | v1.0.9+ | |
| Bundle | [1.1.0](./1.1.0/) | unreleased | `dependencies-v2` |
| Bundle | [1.2.0](./1.2.0/) | unreleased | `persistent-parameters` |

Sometimes you may want to work with a different version of a resource than what is supported by Porter, especially when migrating from one version of Porter to another.
The [schema-check] configuration setting allows you to change how Porter behaves when the schemaVersion of a resource doesn't match Porter's supported version.
Expand Down Expand Up @@ -198,6 +200,7 @@ status:
| parameters.path | true | The path inside the bundle where the parameter value is stored. Either env or path must be specified. |
| parameters.type | true | The parameter type. Allowed values are: string, number, integer, boolean, object, or file. |
| parameters.sensitive | false | Indicates whether this parameter's value is sensitive and should not be logged. |
| parameters.persistent | false | When true, automatically wires up a matching output so the value is remembered across bundle executions. Requires schemaVersion 1.2.0 and the `persistent-parameters` experimental flag. Cannot be combined with `source`, `path`/`env`, or `applyTo`. |
| parameters.source | false | Indicates that the parameter should get its value from an external source. |
| parameters.source.output | true | An output name. The parameter's value is set to output's last value. |
| parameters.source.dependency | false | The name of the dependency that generated the output. If not set, the output must be generated by the current bundle. |
Expand Down
9 changes: 9 additions & 0 deletions docs/content/docs/configuration/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ You may set a default value for a configuration value in the config file, overri
- [Structured Logs](#structured-logs)
- [Dependencies v2](#dependencies-v2)
- [Full control Dockerfile](#full-control-dockerfile)
- [Persistent Parameters](#persistent-parameters)
- [Common Configuration Settings](#common-configuration-settings)
- [Set Current Namespace](#namespace)
- [Output Formatting](#output)
Expand Down Expand Up @@ -322,6 +323,14 @@ When enabled Porter will use the file referenced by `dockerfile` in the Porter m
Ie. Porter will not process `# PORTER_x` placeholders, nor inject any user configuration and `CMD` statements.
It is up to the bundle author to ensure that the contents of the Dockerfile contains the necessary tools for any mixins to function and a layout that can be executed as a Porter bundle.

### Persistent Parameters

The `persistent-parameters` experimental flag enables the `persistent: true` shorthand on parameter definitions.
When set, Porter automatically wires up a matching output so the parameter value is remembered across bundle executions (install → upgrade, install → uninstall), without manually configuring outputs and parameter sources.
Requires `schemaVersion: 1.2.0` in the bundle manifest.

See [Persisting Data Between Bundle Actions](/docs/development/authoring-a-bundle/persisting-data/) for a full guide.

## Common Configuration Settings

Some configuration settings are applicable to many of Porter's commands and to save time you may want to set these values in the configuration file or with environment variables.
Expand Down
111 changes: 102 additions & 9 deletions docs/content/docs/development/authoring-a-bundle/persisting-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,24 @@ When you create a bundle, you often need to preserve data generated during one a
for use in subsequent actions (like `upgrade` or `uninstall`). For example, you might create a virtual
machine during installation and need its ID to delete it during uninstallation.

Porter provides two mechanisms to persist data between bundle actions:
Porter provides three mechanisms to persist data between bundle actions:

1. **Parameter Sources** - Reference outputs from previous actions as parameter values
1. **Parameter Sources** - Full control over wiring outputs back as parameters
2. **State Files** - Automatically persist and restore files between bundle runs
3. **Persistent Parameters** - Shorthand for remembering a simple parameter value across runs (requires schemaVersion 1.2.0 + `persistent-parameters` experimental flag)

This guide helps you understand when to use each approach and how to implement them.

## Quick Comparison

| Feature | Parameter Sources | State Files |
|---------|-------------------|-------------|
| **Access in templates** | ✅ Yes (`bundle.parameters.X`) | ❌ No (file-only access) |
| **User can override** | ✅ Yes | ❌ No |
| **Transparency** | ✅ Visible in parameter sets | ⚠️ Hidden from users |
| **Best for** | IDs, connection strings, config values | Tool state files (terraform.tfstate, kubeconfig) |
| **Data types** | string, number, boolean, file, object | file only |
| Feature | Parameter Sources | State Files | Persistent Parameters |
|---------|-------------------|-------------|-----------------------|
| **Access in templates** | ✅ Yes (`bundle.parameters.X`) | ❌ No (file-only access) | ✅ Yes (`bundle.parameters.X`) |
| **User can override** | ✅ Yes | ❌ No | ✅ Yes |
| **Transparency** | ✅ Visible in parameter sets | ⚠️ Hidden from users | ✅ Visible in parameter sets |
| **Best for** | Outputs captured by steps | Tool state files | Simple remembered inputs |
| **Data types** | string, number, boolean, file, object | file only | string, number, boolean, object |
| **Requires experimental flag** | ❌ No | ❌ No | ✅ Yes (`persistent-parameters`) |

## Approach 1: Parameter Sources

Expand Down Expand Up @@ -223,6 +225,87 @@ Porter saves state files when:
Porter restores state files:
- Before each bundle action runs (except the first install)

## Approach 3: Persistent Parameters (Experimental)

The `persistent: true` shorthand is the simplest way to remember a parameter value across bundle
executions. It is ideal when:

- The value is provided by the user on `install` (e.g., a resource group name, region, or prefix)
- The exact same value must be used on every subsequent `upgrade` and `uninstall`

### Requirements

- `schemaVersion: 1.2.0` in your porter.yaml
- The `persistent-parameters` [experimental feature flag] enabled

```console
export PORTER_EXPERIMENTAL=persistent-parameters
```

### How It Works

When you set `persistent: true` on a parameter, Porter automatically:

1. Stores the parameter value in a file at `/cnab/app/<name>` (same path serves as both parameter
destination and output source)
2. Creates a matching bundle output with the same name
3. Wires the output back to the parameter so subsequent runs receive the saved value without user
input

### Example: Remembering a Resource Group

```yaml
schemaVersion: 1.2.0
name: my-bundle
version: 0.1.0
registry: localhost:5000

mixins:
- exec

parameters:
- name: resource-group
type: string
persistent: true
description: Azure resource group name, remembered after install

install:
- exec:
description: "Deploy to ${bundle.parameters.resource-group}"
command: deploy.sh
arguments:
- "${bundle.parameters.resource-group}"

upgrade:
- exec:
description: "Upgrade"
command: upgrade.sh
arguments:
- "${bundle.parameters.resource-group}" # auto-populated from install

uninstall:
- exec:
description: "Remove resources"
command: teardown.sh
arguments:
- "${bundle.parameters.resource-group}" # auto-populated from install
```

On `install`, the user must provide `--param resource-group=my-rg`. On subsequent `upgrade` and
`uninstall` runs, Porter automatically injects the saved value — no user input required.

Users can still override the value at any time:

```console
porter upgrade mybundle --param resource-group=other-rg
```

### Restrictions

`persistent: true` cannot be combined with `source`, `path`/`env`, or `applyTo`. Use the
[Approach 1: Parameter Sources](#approach-1-parameter-sources) longhand if you need that
flexibility.

## Choosing the Right Approach

Use this decision guide to choose the appropriate mechanism:
Expand All @@ -246,6 +329,14 @@ Use this decision guide to choose the appropriate mechanism:

**Examples:** `terraform.tfstate`, `kubeconfig`, `.terraform.lock.hcl`, SSH keys, certificate files

### Use Persistent Parameters When:

- ✅ The value is a simple user-provided input remembered across runs
- ✅ You want minimal boilerplate (no manual output wiring)
- ✅ You can enable the `persistent-parameters` experimental flag

**Examples:** resource group names, regions, name prefixes, environment names

## Combining Both Approaches

You can use both mechanisms together in the same bundle. For example, you might:
Expand Down Expand Up @@ -351,3 +442,5 @@ variables. If you need template/variable access, use parameter sources instead.
- [State](/docs/bundle/manifest/#state) - State configuration
- [Using Templates](/docs/development/authoring-a-bundle/using-templates/) - Template syntax and functions
- [Working with Dependencies](/docs/development/authoring-a-bundle/working-with-dependencies/) - Sourcing outputs from dependencies

[experimental feature flag]: /docs/configuration/configuration/#experimental
38 changes: 38 additions & 0 deletions pkg/cnab/config-adapter/adapter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1067,6 +1067,44 @@ func TestManifestConverter_generatedMaintainers(t *testing.T) {
}
}

func TestManifestConverter_ToBundle_PersistentParameter(t *testing.T) {
t.Parallel()

c := config.NewTestConfig(t)
c.TestContext.AddTestFileFromRoot("pkg/manifest/testdata/porter-with-persistent-params.yaml", config.Name)
c.SetExperimentalFlags(experimental.FlagPersistentParameters)

ctx := context.Background()
m, err := manifest.LoadManifestFrom(ctx, c.Config, config.Name)
require.NoError(t, err, "could not load manifest")

a := NewManifestConverter(c.Config, m, nil, nil, false)
bun, err := a.ToBundle(ctx)
require.NoError(t, err, "ToBundle failed")

// Parameter exists and is required on install
param, ok := bun.Parameters["resource-group"]
require.True(t, ok, "parameter resource-group should exist")
assert.True(t, param.Required, "persistent parameter should be required")
require.NotNil(t, param.Destination)
assert.Equal(t, "/cnab/app/resource-group", param.Destination.Path)

// Matching output is generated
_, ok = bun.Outputs["resource-group"]
assert.True(t, ok, "output resource-group should be auto-generated")

// Parameter source wires output → parameter
sources, err := bun.ReadParameterSources()
require.NoError(t, err)
ps, ok := sources["resource-group"]
require.True(t, ok, "parameter source for resource-group should exist")
require.Len(t, ps.Sources, 1)
assert.Equal(t, cnab.ParameterSourceTypeOutput, ps.Priority[0])
src, ok := ps.Sources[cnab.ParameterSourceTypeOutput]
require.True(t, ok)
assert.Equal(t, "resource-group", src.(cnab.OutputParameterSource).OutputName)
}

func getMaintainerByName(source []bundle.Maintainer, name string) (bundle.Maintainer, error) {
for _, m := range source {
if m.Name == name {
Expand Down
8 changes: 8 additions & 0 deletions pkg/experimental/experimental.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ const (
// OptimizedBundleBuild is the name of the experimental feature flag for optimized bundle builds
// When enabled, uses .cnab directory as build context with porter-internal-userfiles named context for 54% smaller images
OptimizedBundleBuild = "optimized-bundle-build"

// PersistentParameters is the name of the experimental feature flag for persistent parameters.
PersistentParameters = "persistent-parameters"
)

// FeatureFlags is an enum of possible feature flags
Expand All @@ -30,6 +33,9 @@ const (

// FlagOptimizedBundleBuild gates the optimized bundle build process
FlagOptimizedBundleBuild

// FlagPersistentParameters gates persistent parameter shorthand (persistent: true).
FlagPersistentParameters
)

// ParseFlags converts a list of feature flag names into a bit map for faster lookups.
Expand All @@ -45,6 +51,8 @@ func ParseFlags(flags []string) FeatureFlags {
experimental = experimental | FlagFullControlDockerfile
case OptimizedBundleBuild:
experimental = experimental | FlagOptimizedBundleBuild
case PersistentParameters:
experimental = experimental | FlagPersistentParameters
}
}
return experimental
Expand Down
Loading
Loading