Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changelog/pr-126.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
```release-note:feature
`cmd/list-outputs`: New `list-outputs` subcommand that enumerates all possible output attribute paths (`resource_type.label.attr`) for exported resources. Supports `--depth` (default 1), `--include-resources`, `--exclude-resources`, and `--include-upstream`. Output is newline-delimited on stdout and pipeable directly to `--output-attribute-file`.
```

```release-note:feature
`cmd/export`: New `--output-attribute` (repeatable, glob `*` supported in label position) and `--output-attribute-file` flags that populate `outputs.tf` with Terraform output blocks for specified resource attribute paths.
```

```release-note:enhancement
`resource/pingone_davinci_application`: Added missing `api_key.value` computed attribute to schema definition.
```

```release-note:bug
`formatters/hcl`, `formatters/tfjson`: Computed-only attributes inside nested object blocks (e.g. `api_key.value`) were incorrectly written into resource configuration. The computed skip guard now applies at all nesting levels.
```
73 changes: 73 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Export PingOne resources to Terraform configuration with automatic dependency re
- **Automatic Dependency Resolution**: Generates Terraform references between resources
- **Import Block Generation**: Terraform import blocks to manage existing resources (Terraform 1.5+)
- **Module Structure**: Generates reusable Terraform modules with proper variable scaffolding
- **Output Generation**: Populate `outputs.tf` so parent modules can reference child module resources
- **Dual Mode Operation**: Works as standalone CLI or Ping CLI plugin
- **Two-Environment Authentication**: Isolate credentials from exported resources

Expand Down Expand Up @@ -151,6 +152,24 @@ pingcli-terraformer export --out ./output
| `--exclude-resources` | - | Exclude resources matching glob/regex pattern (repeatable) |
| `--include-upstream` | `false` | Include upstream dependencies of filtered resources |
| `--list-resources` | `false` | List resource addresses and exit |
| `--output-attribute` | - | Add an output block for `resource_type.label.attr` (repeatable; glob `*` supported in label position) |
| `--output-attribute-file` | - | File with one `resource_type.label.attr` path per line (same format as `--output-attribute`) |

### List Outputs Command

Enumerates all possible output attribute paths for exported resources without writing any files. Output is newline-delimited on stdout, suitable for piping or redirecting to a file for use with `--output-attribute-file`.

| Flag | Default | Description |
|------|---------|-------------|
| `--pingone-worker-environment-id` | - | Worker environment ID |
| `--pingone-export-environment-id` | Worker env | Target environment ID |
| `--pingone-worker-client-id` | - | OAuth2 client ID |
| `--pingone-worker-client-secret` | - | OAuth2 client secret |
| `--pingone-region-code` | `NA` | Region: NA, EU, AP, CA, AU, SG |
| `--depth` | `1` | Attribute enumeration depth (1 = top-level only; 2 = one level of nesting) |
| `--include-resources` | - | Include resources matching glob/regex pattern (repeatable) |
| `--exclude-resources` | - | Exclude resources matching glob/regex pattern (repeatable) |
| `--include-upstream` | `false` | Include upstream dependencies of filtered resources |

### Output Formats

Expand Down Expand Up @@ -270,6 +289,60 @@ pingcli-terraformer export \
--out ./output
```

## Generating Module Outputs

When the exported module is used as a child module in a root Terraform configuration, the root module may need to reference resource attributes (e.g. a flow ID) via output blocks. Use `list-outputs` to discover available paths and `--output-attribute` / `--output-attribute-file` to populate `outputs.tf`.

### Discover available output paths

```bash
pingcli-terraformer list-outputs \
--include-resources "pingone_davinci_flow.*" \
--pingone-export-environment-id <uuid>
```

Use `--depth 2` to include nested attributes (e.g. `api_key.value`):

```bash
pingcli-terraformer list-outputs --depth 2 ...
```

### Export with specific output attributes

Using `--output-attribute` directly, with glob support in the label position:

```bash
pingcli-terraformer export \
--output-attribute "pingone_davinci_flow.*.id" \
--output-attribute "pingone_davinci_flow.*.name" \
--out ./output ...
```

### Pipe list-outputs into export

```bash
# Capture all flow output paths, filter to just IDs
pingcli-terraformer list-outputs \
--include-resources "pingone_davinci_flow.*" ... \
| grep '\.id$' > flow-outputs.txt

# Edit flow-outputs.txt as needed, then export
pingcli-terraformer export \
--output-attribute-file flow-outputs.txt \
--out ./output ...
```

The generated `outputs.tf` in the child module will contain one `output` block per path:

```hcl
output "pingone_davinci_flow__pingcli__My-0020-Flow__id" {
description = "The id of pingone_davinci_flow pingcli__My-0020-Flow"
value = pingone_davinci_flow.pingcli__My-0020-Flow.id
}
```

The root module can then reference it as `module.<module_name>.pingone_davinci_flow__pingcli__My-0020-Flow__id`.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development guides, architecture documentation, and how to add new resources.
Expand Down
129 changes: 124 additions & 5 deletions cmd/export.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@
package cmd

import (
"bufio"
"context"
"fmt"
"os"
"path/filepath"
"sort"
"strings"

"github.com/pingidentity/pingcli-plugin-terraformer/definitions"
Expand Down Expand Up @@ -160,6 +163,10 @@ func (c *ExportCommand) Run(args []string, logger grpc.Logger) error {
listResources := flags.Bool("list-resources", false, "List all resource addresses (resource_type.terraform_label) and exit")
includeUpstream := flags.Bool("include-upstream", false, "Automatically include upstream dependencies of filtered resources")

// Output attribute flags
outputAttributes := flags.StringSlice("output-attribute", []string{}, "Add an output block for resource_type.label.attr (repeatable; glob * supported in label position)")
outputAttributeFile := flags.String("output-attribute-file", "", "File with one resource_type.label.attr path per line (same format as --output-attribute)")

// Parse the provided arguments
if err := flags.Parse(args); err != nil {
return err
Expand All @@ -174,12 +181,12 @@ func (c *ExportCommand) Run(args []string, logger grpc.Logger) error {
}

// Execute export
return c.runExport(logger, *workerEnvironmentID, *exportEnvironmentID, *regionCode, *clientID, *clientSecret, *out, *skipDependencies, *moduleDir, *moduleName, *includeImports, *includeValues, *outputFormat, *includeResources, *excludeResources, *listResources, *includeUpstream)
return c.runExport(logger, *workerEnvironmentID, *exportEnvironmentID, *regionCode, *clientID, *clientSecret, *out, *skipDependencies, *moduleDir, *moduleName, *includeImports, *includeValues, *outputFormat, *includeResources, *excludeResources, *listResources, *includeUpstream, *outputAttributes, *outputAttributeFile)
}

// runExport handles API export of all resources from an environment
// All exports now generate Terraform module structure
func (c *ExportCommand) runExport(logger grpc.Logger, workerEnvironmentID, exportEnvironmentID, regionCode, clientID, clientSecret, out string, skipDeps bool, moduleDir string, moduleName string, includeImports bool, includeValues bool, outputFormat string, includeResources []string, excludeResources []string, listResources bool, includeUpstream bool) error {
func (c *ExportCommand) runExport(logger grpc.Logger, workerEnvironmentID, exportEnvironmentID, regionCode, clientID, clientSecret, out string, skipDeps bool, moduleDir string, moduleName string, includeImports bool, includeValues bool, outputFormat string, includeResources []string, excludeResources []string, listResources bool, includeUpstream bool, outputAttributes []string, outputAttributeFile string) error {
// Get credentials from environment variables if not provided via flags
if workerEnvironmentID == "" {
workerEnvironmentID = os.Getenv("PINGCLI_PINGONE_ENVIRONMENT_ID")
Expand Down Expand Up @@ -238,12 +245,12 @@ func (c *ExportCommand) runExport(logger grpc.Logger, workerEnvironmentID, expor
return fmt.Errorf("failed to create API client: %w", err)
}

return c.exportAsModule(ctx, client, logger, skipDeps, includeImports, includeValues, moduleDir, moduleName, out, exportEnvironmentID, outputFormat, includeResources, excludeResources, listResources, includeUpstream)
return c.exportAsModule(ctx, client, logger, skipDeps, includeImports, includeValues, moduleDir, moduleName, out, exportEnvironmentID, outputFormat, includeResources, excludeResources, listResources, includeUpstream, outputAttributes, outputAttributeFile)
}

// exportAsModule uses the schema-driven orchestrator pipeline to export
// resources and generate a Terraform module.
func (c *ExportCommand) exportAsModule(ctx context.Context, client *pingoneplatform.Client, logger grpc.Logger, skipDeps, includeImports, includeValues bool, moduleDir, moduleName, out, environmentID, outputFormat string, includeResources []string, excludeResources []string, listResources bool, includeUpstream bool) error {
func (c *ExportCommand) exportAsModule(ctx context.Context, client *pingoneplatform.Client, logger grpc.Logger, skipDeps, includeImports, includeValues bool, moduleDir, moduleName, out, environmentID, outputFormat string, includeResources []string, excludeResources []string, listResources bool, includeUpstream bool, outputAttributes []string, outputAttributeFile string) error {
outputDir := out
if outputDir == "" {
outputDir = "."
Expand Down Expand Up @@ -422,7 +429,17 @@ func (c *ExportCommand) exportAsModule(ctx context.Context, client *pingoneplatf
})
}

// 7. Build module structure.
// 7. Resolve --output-attribute / --output-attribute-file paths into output blocks.
outputPaths, pathErr := collectOutputPaths(outputAttributes, outputAttributeFile)
if pathErr != nil {
return pathErr
}
var allOutputs []module.Output
if len(outputPaths) > 0 {
allOutputs = buildOutputs(outputPaths, result, logger)
}

// 8. Build module structure.
moduleConfig := module.ModuleConfig{
OutputDir: outputDir,
ModuleDirName: moduleDir,
Expand All @@ -439,6 +456,7 @@ func (c *ExportCommand) exportAsModule(ctx context.Context, client *pingoneplatf
structure := &module.ModuleStructure{
Config: moduleConfig,
Variables: allVariables,
Outputs: allOutputs,
Resources: resources,
ImportBlocks: allImportBlocks,
}
Expand Down Expand Up @@ -498,6 +516,107 @@ func importResourceLabel(data *core.ResourceData, def *schema.ResourceDefinition
return data.ID
}

// outputAttrPath holds a parsed --output-attribute path.
type outputAttrPath struct {
resourceType string
labelPattern string
attrPath string // dot-notation, e.g. "id" or "settings.csp"
}

// parseOutputPath splits "resource_type.label_or_glob.attr" or
// "resource_type.label_or_glob.nested.attr" into its three logical parts.
// The first dot-segment is resource_type, the second is label, and everything
// after the second dot is the attribute path.
func parseOutputPath(raw string) (outputAttrPath, bool) {
parts := strings.SplitN(raw, ".", 3)
if len(parts) < 3 || parts[0] == "" || parts[1] == "" || parts[2] == "" {
return outputAttrPath{}, false
}
return outputAttrPath{
resourceType: parts[0],
labelPattern: parts[1],
attrPath: parts[2],
}, true
}

// collectOutputPaths merges --output-attribute values with lines read from
// --output-attribute-file, deduplicates, and returns the combined slice.
func collectOutputPaths(flags []string, filePath string) ([]string, error) {
seen := make(map[string]struct{})
var paths []string
add := func(p string) {
p = strings.TrimSpace(p)
if p == "" || strings.HasPrefix(p, "#") {
return
}
if _, ok := seen[p]; !ok {
seen[p] = struct{}{}
paths = append(paths, p)
}
}
for _, f := range flags {
add(f)
}
if filePath != "" {
f, err := os.Open(filePath)
if err != nil {
return nil, fmt.Errorf("cannot open --output-attribute-file %q: %w", filePath, err)
}
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
add(scanner.Text())
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("reading --output-attribute-file: %w", err)
}
}
return paths, nil
}

// buildOutputs converts collected path strings into module.Output values by
// matching each (resourceType, labelPattern, attrPath) against exported results.
func buildOutputs(paths []string, result *core.ExportResult, logger grpc.Logger) []module.Output {
// Index results by resource type for O(1) lookup.
byType := make(map[string]*core.ExportedResourceData, len(result.ResourcesByType))
for _, erd := range result.ResourcesByType {
byType[erd.ResourceType] = erd
}

var outputs []module.Output
for _, raw := range paths {
p, ok := parseOutputPath(raw)
if !ok {
_ = logger.Warn(fmt.Sprintf("skipping malformed --output-attribute path %q: expected resource_type.label.attr", raw), nil)
continue
}
erd, found := byType[p.resourceType]
if !found {
_ = logger.Warn(fmt.Sprintf("--output-attribute: resource type %q not found in export results", p.resourceType), nil)
continue
}
matched := 0
for _, rd := range erd.Resources {
ok, err := filepath.Match(strings.ToLower(p.labelPattern), strings.ToLower(rd.Label))
if err != nil || !ok {
continue
}
matched++
name := p.resourceType + "__" + rd.Label + "__" + strings.ReplaceAll(p.attrPath, ".", "__")
outputs = append(outputs, module.Output{
Name: name,
Description: "The " + p.attrPath + " of " + p.resourceType + " " + rd.Label,
Value: p.resourceType + "." + rd.Label + "." + p.attrPath,
})
}
if matched == 0 {
_ = logger.Warn(fmt.Sprintf("--output-attribute: pattern %q matched no %s resources", p.labelPattern, p.resourceType), nil)
}
}
sort.Slice(outputs, func(i, j int) bool { return outputs[i].Name < outputs[j].Name })
return outputs
}

// buildImportID expands the definition's import ID format for a single resource.
func buildImportID(def *schema.ResourceDefinition, rd *core.ResourceData, environmentID string) string {
format := def.Dependencies.ImportIDFormat
Expand Down
Loading
Loading