Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 66 additions & 3 deletions docs/content/en/docs/2_concepts/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: |
## What are Functions in Porch?

**Functions** in Porch are [KRM (Kubernetes Resource Model) functions](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md) -
containerized programs that transform or validate Kubernetes resource manifests within a package's files. Functions are
programs (usually containerized) that transform or validate Kubernetes resource manifests within a package's files. Functions are
declared in a package's Kptfile and executed by Porch when rendering the package.

Functions enable:
Expand All @@ -20,9 +20,70 @@ Functions enable:

For details on how to declare and configure functions in the Kptfile pipeline, see the [kpt functions documentation](https://kpt.dev/book/04-using-functions/).

## Function Configuration

Porch uses **FunctionConfig** custom resources to configure how functions execute.
These configurations determine which executor type should handle the given function images and provide executor-specific settings.

Each FunctionConfig defines:
- **Image and prefixes**: The function image name and optional registry prefixes
- **Executor configurations**: One or more executor types (pod, binary, or go) with associated settings
- **Version-specific settings**: Different executors and configurations for different image tags

Example FunctionConfig:

```yaml
apiVersion: config.porch.kpt.dev/v1alpha1
kind: FunctionConfig
metadata:
name: set-namespace
namespace: porch-fn-system
spec:
image: set-namespace
prefixes:
- ""
- ghcr.io/kptdev/krm-functions-catalog
podExecutor:
tags:
- v0.4.1
timeToLive: 30m
binaryExecutor:
tags:
- v0.4.2
path: set-namespace
goExecutor:
id: set-namespace
tags:
- v0.4
- v0.4.5
```

Most Porch components include an embedded reconciler that watches FunctionConfig resources and populates the component's own internal cache.
This cache determines which executor to use for each function invocation.

## Function Execution in Porch

Porch executes functions through a **function runner** component that calls kpt to orchestrate containerized function execution. The functions run in isolated containers (Kubernetes pods managed by the `function-runner` microservice). Porch passes the package's resources to kpt, which passes the resources on as a [ResourceList](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#resourcelist) to each function in the pipeline in turn. kpt executes the functions sequentially in the order declared in the Kptfile pipeline and passes the function results back to Porch, which stores them in the PackageRevisionResources's `status.renderStatus` field. Execution is triggered automatically following creation or clone of a package revision, update of a package revision, and when a package revision is proposed.
Porch executes functions using multiple executor types, determined by FunctionConfig settings.

**Pod Executor**: Runs functions in isolated containerized pods (the traditional approach):
- Functions execute in Kubernetes pods managed by the `function-runner` microservice
- Configurable with pod templates, resource limits, TTL settings, and maximum parallel executions
- Suitable for functions requiring container isolation or external dependencies
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Pod Executor runs functions in isolated containerized pods (the traditional approach). The functions are executed in Kubernetes pods managed by the function-runner microservice. The executor/functions (not sure which) are configurable with pod templates, resource limits, TTL settings, and maximum parallel executions. This executor is suitable for functions requiring container isolation or external dependencies.


**Binary Executor**: Substitutes specific function image tags with local binary executables:
- Executes pre-built function binaries directly on the host system
- Provides improved performance by avoiding container overhead
- Configured with the file path to the function binary
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Binary Executor substitutes specific function image tags with local binary executables. It executes re-built function binaries directly on the host system and provides improved performance by avoiding container overhead. You can configure this executor with the file path to the function binary.


**Go Executor**: Executes certain functions as native Go function calls within the porch-server process:
- Functions run in-process for maximum efficiency
- Only available for functions integrated as Go libraries
- No container or process overhead
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Go Executor executes certain functions as native Go function calls within the porch-server process. Functions executed under this executer run in-process for maximum efficiency. Go Executor is only available for functions integrated as Go libraries. No containers or process overhead is included(?).


Regardless of executor type, Porch passes the package's resources to [kpt](https://kpt.dev), which passes the resources on as a [ResourceList](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#resourcelist) to each function in the pipeline in turn.
kpt executes the functions sequentially in the order declared in the Kptfile pipeline and passes the function results back to Porch, which stores them in the PackageRevisionResources's `status.renderStatus` field.
Pipeline execution is triggered automatically following creation or clone of a package revision, update of a package revision, and when a package revision is proposed.
kpt passes the function results back to Porch and Porch stores them in the PackageRevisionResources's `status.renderStatus` field.

## When Functions Execute

Expand Down Expand Up @@ -77,7 +138,9 @@ enabling iterative development on incomplete packages.
## Key Points

- Functions are standard KRM functions declared in the Kptfile pipeline (see [kpt functions docs](https://kpt.dev/book/04-using-functions/))
- Porch invokes kpt to execute functions via a function-runner component using containerized execution
- Function execution behavior is configured using FunctionConfig custom resources that specify executor types (pod, binary, or go)
- Most Porch components include an embedded FunctionConfig reconciler that populates internal caches to determine which executor handles each function
- Functions can execute via pod containers, local binaries, or in-process Go calls depending on configuration
- Functions automatically execute during package rendering on Draft package revisions
- Function results are stored in `status.renderStatus` of the PackageRevisionResources view of a package revision
- Published packages are immutable - functions don't re-execute after publication
Expand Down
39 changes: 39 additions & 0 deletions docs/content/en/docs/3_getting_started/installing-porch.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,54 @@ kubectl api-resources | grep porch
You should see Porch API resources:

```bash
functionconfigs config.porch.kpt.dev/v1alpha1 true FunctionConfig
packagerevs config.porch.kpt.dev/v1alpha1 true PackageRev
packagevariants config.porch.kpt.dev/v1alpha1 true PackageVariant
packagevariantsets config.porch.kpt.dev/v1alpha2 true PackageVariantSet
repositories config.porch.kpt.dev/v1alpha1 true Repository
servicetemplates config.porch.kpt.dev/v1alpha1 true ServiceTemplate
packagerevisionresources porch.kpt.dev/v1alpha1 true PackageRevisionResources
packagerevisions porch.kpt.dev/v1alpha1 true PackageRevision
packages porch.kpt.dev/v1alpha1 true PorchPackage
```

Verify that FunctionConfig resources are deployed:

```bash
kubectl get functionconfigs -n porch-fn-system
```

You should see several pre-configured FunctionConfig resources for common KRM functions:

```bash
NAME SERVER APPLIED FNRUNNER APPLIED
apply-replacements 1 1
apply-setters 1 1
create-setters 1 1
ensure-name-substring 1 1
gatekeeper 1 1
kubeconform 1 1
search-replace 1 1
set-annotations 1 1
set-enforcement-action 1 1
set-image 1 1
set-labels 1 1
set-namespace 1 1
starlark 1 1
upsert-resource 1 1
enable-gcp-services 1 1
export-terraform 1 1
generate-folders 1 1
remove-local-config-resources 1 1
set-project-id 1 1
```

**About these new resources:**

- **FunctionConfig**: Defines how KRM functions are executed (pod, binary, or go executors) and replaces the older ConfigMap-based configuration approach. The function-runner and porch-server both include embedded FunctionConfig reconcilers that populate an internal cache used to determine which executor to use and with what configuration.

- **ServiceTemplate**: Defines pod and service templates for pod-based function execution. These templates are referenced by FunctionConfig resources that use the pod executor.

## Troubleshooting

### Pods not starting
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: |
Overview of function runner functionality and detailed documentation pages.
---

The Function Runner provides three core functional areas that work together to execute KRM functions in isolated environments:
The Function Runner provides four core functional areas that work together to execute KRM functions in isolated environments:

## Functional Areas

Expand All @@ -22,6 +22,21 @@ Executes KRM functions through pluggable evaluator strategies:

For detailed architecture and process flows, see [Function Evaluation]({{% relref "/docs/5_architecture_and_components/function-runner/functionality/function-evaluation.md" %}}).

### Function Configuration Management

Provides declarative configuration of function executors through Kubernetes CRDs:
- **FunctionConfig Reconciler**: Embedded controller watches FunctionConfig resources and maintains internal cache
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **FunctionConfig Reconciler**: Embedded controller watches FunctionConfig resources and maintains internal cache
- **FunctionConfig Reconciler**: Embedded controller that watches FunctionConfig resources and maintains internal cache

- **Executor Selection Cache**: Maps function images to executor types and configuration settings
- **Pod Executor Configuration**: Template overrides, TTL settings, and maximum parallel executions per function
- **Binary Executor Configuration**: Path mapping for substituting container images with local executables
- **Go Executor Configuration**: Function ID registration for native go function execution
- **Image Prefix Matching**: Supports multiple image prefixes and tags per function configuration
- **Template Customization**: Per-function pod and service template overrides including security context, resources, and environment variables

For detailed configuration examples, see [Function Runner Configuration]({{% relref "/docs/6_configuration_and_deployments/configurations/components/function-runner-config/_index.md" %}}).

For integration with executor selection, see [Function Runner Interactions]({{% relref "/docs/5_architecture_and_components/function-runner/interactions.md" %}}).

### Pod Lifecycle Management

Manages function execution pods with caching and garbage collection:
Expand All @@ -32,6 +47,7 @@ Manages function execution pods with caching and garbage collection:
- **TTL-Based Caching**: Reuses pods with configurable expiration and extension on use
- **Garbage Collection**: Periodic cleanup of expired pods and failed pod handling
- **Pod Warming**: Pre-creates pods for frequently-used functions
- **Template Overrides**: Applies FunctionConfig-specified customizations to pod and service templates

For detailed architecture and process flows, see [Pod Lifecycle Management]({{% relref "/docs/5_architecture_and_components/function-runner/functionality/pod-lifecycle-management.md" %}}).

Expand All @@ -52,7 +68,20 @@ For detailed architecture and process flows, see [Image and Registry Management]
```
┌─────────────────────────────────────────────────────────┐
│ Function Runner Service │
│ │
│ ┌──────────────────┐ │
│ │ Function │ │
│ │ Configuration │ │
│ │ Management │ │
│ │ │ │
│ │ • FunctionConfig │ │
│ │ Reconciler │ │
│ │ • Executor Cache │ │
│ │ • Image Prefix │ │
│ │ Matching │ │
│ └──────────────────┘ │
│ │ │
│ ┌────────────┴─────────────┐ │
│ ↓ ↓ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Function │ │ Pod │ │
│ │ Evaluation │ ───> │ Lifecycle │ │
Expand Down Expand Up @@ -84,15 +113,18 @@ For detailed architecture and process flows, see [Image and Registry Management]

**Integration flow:**
1. **Function Evaluation** receives gRPC request from Task Handler
2. **Multi-Evaluator** tries executable evaluator first (fast path)
3. **If NotFound**, falls back to pod evaluator (container execution)
4. **Pod Lifecycle Management** checks pod cache for existing pod
5. **If cache miss**, creates new pod with wrapper server via Pod Manager
6. **Image & Registry Management** resolves image metadata and authentication
7. **Pod Manager** creates pod with image pull secrets and service frontend
8. **Pod Cache Manager** stores pod with TTL for reuse
9. **Function Evaluation** connects to pod via service and executes function
10. **Wrapper Server** executes function binary and returns structured results
11. **Garbage Collection** periodically removes expired pods from cache

2. **Function Configuration Management** queries cache for function-specific configuration
3. **Multi-Evaluator** selects appropriate evaluator based on FunctionConfig settings
4. **If binary executor configured**, executes local function binary (fast path)
5. **If go executor configured**, invokes registered native go function
6. **If pod executor configured or no match**, falls back to pod evaluator (container execution)
7. **Pod Lifecycle Management** checks pod cache for existing pod matching configuration
8. **If cache miss**, creates new pod with FunctionConfig template overrides via Pod Manager
9. **Image & Registry Management** resolves image metadata and authentication
10. **Pod Manager** creates pod with image pull secrets and service frontend
11. **Pod Cache Manager** stores pod with FunctionConfig-specified TTL for reuse
12. **Function Evaluation** connects to pod via service and executes function
13. **Wrapper Server** executes function binary and returns structured results
14. **Garbage Collection** periodically removes expired pods based on TTL settings

Each functional area is documented in detail on its own page with architecture diagrams, process flows, and implementation specifics.
Loading
Loading