PR Controller PoC

[efiacor]

Title

v1alpha2 PackageRevision Controller & Supporting Infrastructure

---

Description

Overview

Introduces a new controller-runtime reconciler for v1alpha2 PackageRevision CRDs, decoupling package lifecycle management (source execution, rendering, lifecycle transitions) from the API server's synchronous request path into an async reconciliation loop.

Opt-in via --repositories.create-v1alpha2-rpkg=true on the controller and porch.kpt.dev/v1alpha2-migration: "true" annotation on Repository CRs.

1. PackageRevision Controller (controllers/packagerevisions/)

~1470 lines production code, ~3000 lines tests. Watches v1alpha2 PackageRevision CRDs and handles:

• Source execution (source.go): init, clone, copy, upgrade — writes resources to git via the shared cache, tracked by status.creationSource.
• Async rendering (render.go): KRM function pipelines via fn-runner (gRPC). Triggered by render-request annotation (PRR push path) or post-source-execution. Bounded concurrency via renderLimiter. Supports push-on-render-failure.
• Lifecycle transitions: Draft → Proposed → Published and Published ↔ DeletionProposed, with publish metadata (revision, publishedBy, publishedAt).
• Status (status.go): SSA with distinct field managers for controller state, render state, and Kptfile-derived fields. Reports Ready and Rendered conditions.
• Labels (labels.go): Manages latest-revision, ensuring only the highest published revision is marked latest.
• Field indexes (fieldindex.go): controller-runtime indexes for efficient lookup by repository, package, workspace.
• Merge keys (mergekey.go): Local copy of merge-key utilities to avoid v1alpha1-coupled imports.
• Integration tests (integration/): envtest-based, covering the full reconcile loop.

Shared cache: The repo and PR controllers run in the same process, sharing a single cache instance. The repo controller creates it at startup; the PR controller accesses it via the ContentCache wrapper — avoiding duplicated git state and ensuring a consistent view.

Function runtime: Unlike the API server (synchronous render inside the request), the PR controller instantiates its own MultiFunctionRuntime (builtin-first, gRPC fallback) directly in the controller binary, configured via FUNCTION_RUNNER_ADDRESS. If unavailable, rendering is skipped.

Wired in controllers/main.go with explicit ordering — repo controller first (initializes cache), then PR controller (receives cache, credential resolvers, function runtime).

2. Content Cache & Interfaces (pkg/cache/contentcache/, pkg/repository/content.go)

Version-neutral abstraction layer between the PR controller and the existing cache, avoiding v1alpha1 type coupling:

• ContentCache: GetPackageContent, UpdateLifecycle, CreateNewDraft, CreateDraftFromExisting, CloseDraft, DeletePackage — all plain strings.
• PackageContent: Read-only access to resources, Kptfile, locks, commit info.
• PackageRevisionDraftSlim: Minimal draft interface for resource/lifecycle updates.
• ExternalPackageFetcher: Fetches from external git repos (clone-from-upstream) with credential/CA resolution.
• Wrappers (wrappers.go): Adapt existing PackageRevision/PackageRevisionDraft to the new interfaces.
• contentcache.go: Backed by cachetypes.Cache, handles repo resolution, package lookup, lifecycle state machine.

3. Existing Interface Changes (pkg/repository/repository.go)

• GetCommitInfo() (time.Time, string): Added to PackageRevision interface. Implemented across git, OCI, fake, crcache, dbcache. Git backend now captures publish metadata (time + user email) at ClosePackageRevisionDraft.
• IsLatestRevision() bool: Promoted from unexported hasLatestRevisionInfo to a first-class interface method — eliminates type assertions in label generation.

4. v1alpha2 API Types (api/porch/v1alpha2/)

• Status: Added ObservedGeneration, SSA-compatible Conditions (listType=map, listMapKey=type), PublishedAt → *metav1.Time, Revision always serialized.
• Constants: AnnotationRenderRequest, PushOnFnRenderFailureKey, ConditionRendered, ReasonRendered, ReasonRenderFailed.
• Kptfile helpers (kptdata_conversion.go): KptfileToPackageConditions, KptfileToReadinessGates, KptfileToPackageMetadata, KptUpstreamLockToLocator — shared by repo controller (CRD sync) and PR controller (status).
• CRD regenerated.

5. Repository Controller — CRD Sync (controllers/repositories/)

• crdsync.go: When enabled, creates/updates/deletes v1alpha2 PackageRevision CRDs to match discovered packages. SSA with separate spec/status applies. Ownership split: repo controller owns identity/lock fields; PR controller owns lifecycle/render/publish fields.
• sync.go: Calls syncPackageRevisionCRDs post-sync. CRD failures are non-fatal.
• RBAC: Added porch.kpt.dev/packagerevisions and packagerevisions/status permissions.

6. API Server Changes (pkg/registry/porch/, pkg/engine/)

• v1alpha2 filtering (packagecommon.go): v1alpha2-annotated repos excluded from v1alpha1 list/get/update/watch. v1alpha2FilteringWatcher suppresses watch events.
• PRR handler (packagerevisionresources.go): v1alpha2 repos use UpdatePackageResourcesWithoutRender (no sync render), then patch render-request annotation to trigger async render. New getRepoPkgRevForResources handles both API versions.
• Engine (engine.go): UpdatePackageResourcesWithoutRender — writes resources and closes draft without render pipeline.

7. porchctl CLI (pkg/cli/commands/rpkg/, internal/cliutils/)

• --api-version flag: Persistent on rpkg, also reads PORCHCTL_API_VERSION env. Defaults to v1alpha1.
• Version dispatch (dispatch.go): WrapVersionDispatch routes PreRunE/RunE to v1alpha2 implementations when flagged.
• v1alpha2 subcommands: Each rpkg command (get, clone, copy, init, del, approve, propose, proposedelete, reject, upgrade) has v1alpha2.go + v1alpha2_test.go.
• v1alpha2 client (client_v1alpha2.go): Custom REST mapper routing PackageRevision to porch.kpt.dev/v1alpha2 (CRD) while keeping PackageRevisionResources at v1alpha1 (API server).
• repo reg: Supports setting the v1alpha2 migration annotation on registration.

8. Deployment & Build

• Make targets: run-in-kind-v1alpha2, run-in-kind-v1alpha2-no-controller (local dev).
• Blueprint script: --create-v1alpha2-rpkg flag — installs CRD, flips controller flag, adds packagerevisions reconciler.
• Controller deployment: --repositories.create-v1alpha2-rpkg=false default arg.
• Mocks: Updated CaDEngine, PackageRevision; new ContentCache, ExternalPackageFetcher, PackageContent.

9. porchconfig Annotation (api/porchconfig/v1alpha1/types.go)

AnnotationKeyV1Alpha2Migration / AnnotationValueMigrationEnabled — per-repository opt-in gating CRD sync and v1alpha2 filtering.

---

Related Issue(s)

• Related#
  EPIC: Moving PackageRevision to CRD #811
  PIP: PackageRevision CRD Migration #890

---

Type of Change

• Bug fix
• New feature
• Enhancement
• Refactor
• Documentation
• Tests
• Other: ________

---

Checklist

• Code follows project style guidelines
• Self-reviewed changes
• Tests added/updated
• Documentation added/updated
• All tests and gating checks pass

---

Testing Instructions

1. Deploy with v1alpha2 enabled:

   make run-in-kind-v1alpha2

   This builds all images, deploys to kind with DB cache, installs the v1alpha2 PackageRevision CRD, enables the packagerevisions reconciler, and sets --repositories.create-v1alpha2-rpkg=true on the controller.

2. Register a v1alpha2-managed repository:

   porchctl repo reg https://github.com/<org>/<repo>.git --namespace default --name my-repo --v1alpha2

   This sets the porch.kpt.dev/v1alpha2-migration: "true" annotation on the Repository CR. The repo controller will sync packages and create v1alpha2 PackageRevision CRDs. Verify with:

   kubectl get packagerevisions.v1alpha2.porch.kpt.dev -n default

3. Use porchctl with v1alpha2:

   porchctl rpkg get --namespace default --api-version=v1alpha2
   porchctl rpkg clone <upstream> <name> --namespace default --repository my-repo --api-version=v1alpha2
   porchctl rpkg propose <name> --namespace default --api-version=v1alpha2
   porchctl rpkg approve <name> --namespace default --api-version=v1alpha2

   Or set export PORCHCTL_API_VERSION=v1alpha2 to avoid repeating the flag.

4. Local controller development (run controller outside the cluster):

   make run-in-kind-v1alpha2-no-controller

   Then run the controller locally with appropriate kubeconfig and FUNCTION_RUNNER_ADDRESS pointing to the in-cluster fn-runner.

   VSCode: Use the Launch Controllers (repositories + packagerevisions v1alpha2) debug configuration in .vscode/launch.json. It starts both the repositories and packagerevisions reconcilers with --repositories.create-v1alpha2-rpkg=true, DB cache env vars, and FUNCTION_RUNNER_ADDRESS pre-configured. Adjust DB_HOST and FUNCTION_RUNNER_ADDRESS IPs if your kind network differs.

---

Additional Notes

• Known issues:

  • Integration tests use mocked cache — source execution, rendering, and label management are covered by unit tests only, not yet by envtest integration tests.
  • The v1alpha2FilteringWatcher in packagecommon.go does a live repo lookup per watch event — acceptable at current scale but may need caching if watch volume grows.

• Further improvements:

  • Add a degraded condition on the Repository CR (e.g. PackageRevisionSyncComplete=False) to surface partial CRD sync failures.
  • Replace r.cad.ListPackageRevisions in getRepoPkgRevForResources with direct cache access once the engine dependency can be removed.
  • End-to-end integration tests with a real git repo and fn-runner.

• Review notes:

  • The feature is fully opt-in — no behaviour change for existing v1alpha1 repos.
  • mergekey.go is a deliberate duplication from pkg/task to avoid pulling in v1alpha1 types into the controller package.
  • NewGRPCFunctionRuntime was renamed from newGRPCFunctionRuntime (exported) to allow the controller binary to instantiate its own runtime.

🤖 AI Assistance Disclosure

This pull request was created with the assistance of generative AI tools. (AmazonQ Developer VSCode plugin)
All code and changes have been reviewed and validated by the author.

[netlify]

✅ Deploy Preview for porch ready!

Name	Link
🔨 Latest commit	4b66cd2
🔍 Latest deploy log	https://app.netlify.com/projects/porch/deploys/69f21b2788039600084aec37
😎 Deploy Preview	https://deploy-preview-514--porch.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[nagygergo]

Some way through this. Not sure how much time I'll have next week, so posting a partial review.

[nagygergo]

If setting ownership for all the parameters, then how clients will be able to do SSA on their own?

[efiacor]

Zero-value fields are excluded from the SSA payload. The repo controller only claims ownership of fields it explicitly sets — lifecycle, source, readinessGates etc. are omitted and remain unowned by this field manager.

[sonarqubecloud]

Quality Gate failed

Failed conditions
7.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud