-
Notifications
You must be signed in to change notification settings - Fork 36
Add documentation for FunctionConfig CRD and reconciler #976
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,7 +9,7 @@ description: | | |
| ## What are Functions in Porch? | ||
|
|
||
| **Functions** in Porch are [KRM (Kubernetes Resource Model) functions](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md) - | ||
| containerized programs that transform or validate Kubernetes resource manifests within a package's files. Functions are | ||
| programs (usually containerized) that transform or validate Kubernetes resource manifests within a package's files. Functions are | ||
| declared in a package's Kptfile and executed by Porch when rendering the package. | ||
|
|
||
| Functions enable: | ||
|
|
@@ -20,9 +20,70 @@ Functions enable: | |
|
|
||
| For details on how to declare and configure functions in the Kptfile pipeline, see the [kpt functions documentation](https://kpt.dev/book/04-using-functions/). | ||
|
|
||
| ## Function Configuration | ||
|
|
||
| Porch uses **FunctionConfig** custom resources to configure how functions execute. | ||
| These configurations determine which executor type should handle the given function images and provide executor-specific settings. | ||
|
|
||
| Each FunctionConfig defines: | ||
| - **Image and prefixes**: The function image name and optional registry prefixes | ||
| - **Executor configurations**: One or more executor types (pod, binary, or go) with associated settings | ||
| - **Version-specific settings**: Different executors and configurations for different image tags | ||
|
|
||
| Example FunctionConfig: | ||
|
|
||
| ```yaml | ||
| apiVersion: config.porch.kpt.dev/v1alpha1 | ||
| kind: FunctionConfig | ||
| metadata: | ||
| name: set-namespace | ||
| namespace: porch-fn-system | ||
| spec: | ||
| image: set-namespace | ||
| prefixes: | ||
| - "" | ||
| - ghcr.io/kptdev/krm-functions-catalog | ||
| podExecutor: | ||
| tags: | ||
| - v0.4.1 | ||
| timeToLive: 30m | ||
| binaryExecutor: | ||
| tags: | ||
| - v0.4.2 | ||
| path: set-namespace | ||
| goExecutor: | ||
| id: set-namespace | ||
| tags: | ||
| - v0.4 | ||
| - v0.4.5 | ||
| ``` | ||
|
|
||
| Most Porch components include an embedded reconciler that watches FunctionConfig resources and populates the component's own internal cache. | ||
| This cache determines which executor to use for each function invocation. | ||
|
|
||
| ## Function Execution in Porch | ||
|
|
||
| Porch executes functions through a **function runner** component that calls kpt to orchestrate containerized function execution. The functions run in isolated containers (Kubernetes pods managed by the `function-runner` microservice). Porch passes the package's resources to kpt, which passes the resources on as a [ResourceList](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#resourcelist) to each function in the pipeline in turn. kpt executes the functions sequentially in the order declared in the Kptfile pipeline and passes the function results back to Porch, which stores them in the PackageRevisionResources's `status.renderStatus` field. Execution is triggered automatically following creation or clone of a package revision, update of a package revision, and when a package revision is proposed. | ||
| Porch executes functions using multiple executor types, determined by FunctionConfig settings. | ||
|
|
||
| **Pod Executor**: Runs functions in isolated containerized pods (the traditional approach): | ||
| - Functions execute in Kubernetes pods managed by the `function-runner` microservice | ||
| - Configurable with pod templates, resource limits, TTL settings, and maximum parallel executions | ||
| - Suitable for functions requiring container isolation or external dependencies | ||
|
|
||
| **Binary Executor**: Substitutes specific function image tags with local binary executables: | ||
| - Executes pre-built function binaries directly on the host system | ||
| - Provides improved performance by avoiding container overhead | ||
| - Configured with the file path to the function binary | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Binary Executor substitutes specific function image tags with local binary executables. It executes re-built function binaries directly on the host system and provides improved performance by avoiding container overhead. You can configure this executor with the file path to the function binary. |
||
|
|
||
| **Go Executor**: Executes certain functions as native Go function calls within the porch-server process: | ||
| - Functions run in-process for maximum efficiency | ||
| - Only available for functions integrated as Go libraries | ||
| - No container or process overhead | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Go Executor executes certain functions as native Go function calls within the porch-server process. Functions executed under this executer run in-process for maximum efficiency. Go Executor is only available for functions integrated as Go libraries. No containers or process overhead is included(?). |
||
|
|
||
| Regardless of executor type, Porch passes the package's resources to [kpt](https://kpt.dev), which passes the resources on as a [ResourceList](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#resourcelist) to each function in the pipeline in turn. | ||
| kpt executes the functions sequentially in the order declared in the Kptfile pipeline and passes the function results back to Porch, which stores them in the PackageRevisionResources's `status.renderStatus` field. | ||
| Pipeline execution is triggered automatically following creation or clone of a package revision, update of a package revision, and when a package revision is proposed. | ||
| kpt passes the function results back to Porch and Porch stores them in the PackageRevisionResources's `status.renderStatus` field. | ||
|
|
||
| ## When Functions Execute | ||
|
|
||
|
|
@@ -77,7 +138,9 @@ enabling iterative development on incomplete packages. | |
| ## Key Points | ||
|
|
||
| - Functions are standard KRM functions declared in the Kptfile pipeline (see [kpt functions docs](https://kpt.dev/book/04-using-functions/)) | ||
| - Porch invokes kpt to execute functions via a function-runner component using containerized execution | ||
| - Function execution behavior is configured using FunctionConfig custom resources that specify executor types (pod, binary, or go) | ||
| - Most Porch components include an embedded FunctionConfig reconciler that populates internal caches to determine which executor handles each function | ||
| - Functions can execute via pod containers, local binaries, or in-process Go calls depending on configuration | ||
| - Functions automatically execute during package rendering on Draft package revisions | ||
| - Function results are stored in `status.renderStatus` of the PackageRevisionResources view of a package revision | ||
| - Published packages are immutable - functions don't re-execute after publication | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -6,7 +6,7 @@ description: | | |||||
| Overview of function runner functionality and detailed documentation pages. | ||||||
| --- | ||||||
|
|
||||||
| The Function Runner provides three core functional areas that work together to execute KRM functions in isolated environments: | ||||||
| The Function Runner provides four core functional areas that work together to execute KRM functions in isolated environments: | ||||||
|
|
||||||
| ## Functional Areas | ||||||
|
|
||||||
|
|
@@ -22,6 +22,21 @@ Executes KRM functions through pluggable evaluator strategies: | |||||
|
|
||||||
| For detailed architecture and process flows, see [Function Evaluation]({{% relref "/docs/5_architecture_and_components/function-runner/functionality/function-evaluation.md" %}}). | ||||||
|
|
||||||
| ### Function Configuration Management | ||||||
|
|
||||||
| Provides declarative configuration of function executors through Kubernetes CRDs: | ||||||
| - **FunctionConfig Reconciler**: Embedded controller watches FunctionConfig resources and maintains internal cache | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| - **Executor Selection Cache**: Maps function images to executor types and configuration settings | ||||||
| - **Pod Executor Configuration**: Template overrides, TTL settings, and maximum parallel executions per function | ||||||
| - **Binary Executor Configuration**: Path mapping for substituting container images with local executables | ||||||
| - **Go Executor Configuration**: Function ID registration for native go function execution | ||||||
| - **Image Prefix Matching**: Supports multiple image prefixes and tags per function configuration | ||||||
| - **Template Customization**: Per-function pod and service template overrides including security context, resources, and environment variables | ||||||
|
|
||||||
| For detailed configuration examples, see [Function Runner Configuration]({{% relref "/docs/6_configuration_and_deployments/configurations/components/function-runner-config/_index.md" %}}). | ||||||
|
|
||||||
| For integration with executor selection, see [Function Runner Interactions]({{% relref "/docs/5_architecture_and_components/function-runner/interactions.md" %}}). | ||||||
|
|
||||||
| ### Pod Lifecycle Management | ||||||
|
|
||||||
| Manages function execution pods with caching and garbage collection: | ||||||
|
|
@@ -32,6 +47,7 @@ Manages function execution pods with caching and garbage collection: | |||||
| - **TTL-Based Caching**: Reuses pods with configurable expiration and extension on use | ||||||
| - **Garbage Collection**: Periodic cleanup of expired pods and failed pod handling | ||||||
| - **Pod Warming**: Pre-creates pods for frequently-used functions | ||||||
| - **Template Overrides**: Applies FunctionConfig-specified customizations to pod and service templates | ||||||
|
|
||||||
| For detailed architecture and process flows, see [Pod Lifecycle Management]({{% relref "/docs/5_architecture_and_components/function-runner/functionality/pod-lifecycle-management.md" %}}). | ||||||
|
|
||||||
|
|
@@ -52,7 +68,20 @@ For detailed architecture and process flows, see [Image and Registry Management] | |||||
| ``` | ||||||
| ┌─────────────────────────────────────────────────────────┐ | ||||||
| │ Function Runner Service │ | ||||||
| │ │ | ||||||
| │ ┌──────────────────┐ │ | ||||||
| │ │ Function │ │ | ||||||
| │ │ Configuration │ │ | ||||||
| │ │ Management │ │ | ||||||
| │ │ │ │ | ||||||
| │ │ • FunctionConfig │ │ | ||||||
| │ │ Reconciler │ │ | ||||||
| │ │ • Executor Cache │ │ | ||||||
| │ │ • Image Prefix │ │ | ||||||
| │ │ Matching │ │ | ||||||
| │ └──────────────────┘ │ | ||||||
| │ │ │ | ||||||
| │ ┌────────────┴─────────────┐ │ | ||||||
| │ ↓ ↓ │ | ||||||
| │ ┌──────────────────┐ ┌──────────────────┐ │ | ||||||
| │ │ Function │ │ Pod │ │ | ||||||
| │ │ Evaluation │ ───> │ Lifecycle │ │ | ||||||
|
|
@@ -84,15 +113,18 @@ For detailed architecture and process flows, see [Image and Registry Management] | |||||
|
|
||||||
| **Integration flow:** | ||||||
| 1. **Function Evaluation** receives gRPC request from Task Handler | ||||||
| 2. **Multi-Evaluator** tries executable evaluator first (fast path) | ||||||
| 3. **If NotFound**, falls back to pod evaluator (container execution) | ||||||
| 4. **Pod Lifecycle Management** checks pod cache for existing pod | ||||||
| 5. **If cache miss**, creates new pod with wrapper server via Pod Manager | ||||||
| 6. **Image & Registry Management** resolves image metadata and authentication | ||||||
| 7. **Pod Manager** creates pod with image pull secrets and service frontend | ||||||
| 8. **Pod Cache Manager** stores pod with TTL for reuse | ||||||
| 9. **Function Evaluation** connects to pod via service and executes function | ||||||
| 10. **Wrapper Server** executes function binary and returns structured results | ||||||
| 11. **Garbage Collection** periodically removes expired pods from cache | ||||||
|
|
||||||
| 2. **Function Configuration Management** queries cache for function-specific configuration | ||||||
| 3. **Multi-Evaluator** selects appropriate evaluator based on FunctionConfig settings | ||||||
| 4. **If binary executor configured**, executes local function binary (fast path) | ||||||
| 5. **If go executor configured**, invokes registered native go function | ||||||
| 6. **If pod executor configured or no match**, falls back to pod evaluator (container execution) | ||||||
| 7. **Pod Lifecycle Management** checks pod cache for existing pod matching configuration | ||||||
| 8. **If cache miss**, creates new pod with FunctionConfig template overrides via Pod Manager | ||||||
| 9. **Image & Registry Management** resolves image metadata and authentication | ||||||
| 10. **Pod Manager** creates pod with image pull secrets and service frontend | ||||||
| 11. **Pod Cache Manager** stores pod with FunctionConfig-specified TTL for reuse | ||||||
| 12. **Function Evaluation** connects to pod via service and executes function | ||||||
| 13. **Wrapper Server** executes function binary and returns structured results | ||||||
| 14. **Garbage Collection** periodically removes expired pods based on TTL settings | ||||||
|
|
||||||
| Each functional area is documented in detail on its own page with architecture diagrams, process flows, and implementation specifics. | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Pod Executor runs functions in isolated containerized pods (the traditional approach). The functions are executed in Kubernetes pods managed by the
function-runnermicroservice. The executor/functions (not sure which) are configurable with pod templates, resource limits, TTL settings, and maximum parallel executions. This executor is suitable for functions requiring container isolation or external dependencies.