tests: Add provisioned-environment E2E test tier - #141
Open
samir-gandhi wants to merge 2 commits into
Open
Conversation
Adds terraform-test-data/ (hand-authored Terraform fixtures per resource), tools/tf-regression-provision (creates/destroys a throwaway PingOne environment via Terraform itself, using org-admin credentials scoped to this run's lifecycle), and a `make e2e` entry point that applies fixtures into that environment and runs the existing base-vs-PR export/compare pipeline against it. Additive to tests/regression/, which continues comparing export output against a static environment unchanged. Closes #140.
Regression Test Results
✅ No regressions detectedAll export configurations produced compatible output. Generated by regression workflow • View run |
Regression Test Results
✅ No regressions detectedAll export configurations produced compatible output. Generated by regression workflow • View run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
terraform-test-data/— hand-authored Terraform fixtures, starting with one representative resource (sso/population).pingone_environmentitself is a fixture resource interraform-test-data/root, so environment creation goes through the same Terraform/provider path as everything else.tools/tf-regression-provision— thin CLI wrapper aroundterraform apply/destroyplus output parsing. Holds no PingOne API logic itself; environment lifecycle and fixture provisioning are entirely expressed as Terraform, authenticated with a single org-admin credential scoped to this run.make e2e(backed bytests/regression-provisioned/run-local.sh), which provisions a throwaway environment, applies the fixtures, runs the existing base-vs-PR export/compare pipeline (tools/regression-compare,tests/regression/matrix.json) against it, and tears the environment down unconditionally on exit..gitignorefor local/ephemeral Terraform artifacts (.terraform/, lock file, state,terraform.tfvars.json) and to stop the existing blanket*.tfignore (meant for generated export output) from swallowing the new hand-authoredterraform-test-data/**/*.tffixtures.This is additive to
tests/regression/, which is unchanged and continues comparing export output between binary versions against its existing static environment.Closes #140.
Test plan
go build ./tools/tf-regression-provision/...andgo vet ./tools/tf-regression-provision/...terraform validateandterraform fmt -check -recursiveonterraform-test-data/bash -n tests/regression-provisioned/run-local.shgo mod tidy— nogo.mod/go.sumchanges (no new dependencies)make e2elive end-to-end against a real (non-prod) PingOne org: provisioned an environment, applied thesso/populationfixture, ran all 5 matrix entries fromtests/regression/matrix.json(base vs. PR export, 0 breaking / 0 acceptable diffs on each), then tore the environment down. Confirmed via a follow-up API read that the environment no longer exists after teardown.Notes for reviewers
PINGCLI_PINGONE_ORGADMIN_CLIENT_ID,PINGCLI_PINGONE_ORGADMIN_CLIENT_SECRET,PINGCLI_PINGONE_ORGADMIN_ENVIRONMENT_ID,PINGCLI_PINGONE_ORGADMIN_REGION_CODE,PINGCLI_PINGONE_ORGADMIN_LICENSE_ID. Not wired into CI in this PR — local-only viamake e2epending a decision on scheduling/cost with the org owner.terraform-test-data/currently covers only 1 of the ~23 currently-supported resource types; follow-up PRs will expand coverage in small, dependency-ordered batches (see the tracking issue).