-
Notifications
You must be signed in to change notification settings - Fork 561
Add a new name resolution method: structuredformat #4085
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
aa23b2d
Add a new name resolution method: structuredformat.
seal90 40d44f8
Merge branch 'dapr:main' into config-string
seal90 1b7b343
fix checkstyle error
seal90 4c7daec
Merge branch 'dapr:main' into config-string
seal90 b268a9b
Merge branch 'config-string' of https://github.com/seal90/components-…
seal90 550b60f
Merge branch 'main' into config-string
sicoyle a5fd97b
optimize structuredformat code and docs
seal90 9ce8641
Merge branch 'config-string' of https://github.com/seal90/components-…
seal90 95edccd
Merge branch 'main' into config-string
cicoyle File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # Structured Format Name Resolution | ||
|
|
||
| The **Structured Format** name resolver enables you to explicitly define service instances using structured configuration in **JSON or YAML**, either as inline strings or external files. It is designed for scenarios where service topology is **static and known in advance**, such as: | ||
|
|
||
| - Local development and testing | ||
| - Integration or end-to-end test environments | ||
| - Edge deployments | ||
|
|
||
| ## Configuration Format | ||
|
|
||
| To enable the resolver, configure it in your Dapr `Configuration` resource: | ||
|
|
||
| ```yaml | ||
| apiVersion: dapr.io/v1alpha1 | ||
| kind: Configuration | ||
| metadata: | ||
| name: appconfig | ||
| spec: | ||
| nameResolution: | ||
| component: "structuredformat" | ||
| configuration: | ||
| structuredType: "json" # or "yaml", "jsonFile", "yamlFile" | ||
| stringValue: '{"appInstances":{"myapp":[{"ipv4":"127.0.0.1","port":4433}]}}' | ||
| ``` | ||
|
|
||
| ## Spec configuration fields | ||
|
|
||
| | Field | Required? | Description | Example | | ||
| |------------------|-----------|-----------------------------------------------------------------------------|---------| | ||
| | `structuredType` | Yes | Format and source type. Must be one of: `json`, `yaml`, `jsonFile`, `yamlFile` | `json` | | ||
| | `stringValue` | Conditional | Required when `structuredType` is `json` or `yaml` | `{"appInstances":{"myapp":[{"ipv4":"127.0.0.1","port":4433}]}}` | | ||
| | `filePath` | Conditional | Required when `structuredType` is `jsonFile` or `yamlFile` | `/etc/dapr/services.yaml` | | ||
|
|
||
| > **Important**: Only one of `stringValue` or `filePath` should be provided, based on `structuredType`. | ||
|
|
||
| ## `appInstances` Schema | ||
|
|
||
| The configuration must contain a top-level `appInstances` object that maps **service IDs** to **lists of address instances**. | ||
|
|
||
| ### Supported Address Fields | ||
|
|
||
| | Field | Type | Required? | Description | | ||
| |----------|--------|-----------|-------------| | ||
| | `domain` | string | No | Hostname or FQDN (e.g., `"api.example.com"`). Highest priority. | | ||
| | `ipv4` | string | No | IPv4 address in dotted-decimal format (e.g., `"192.168.1.10"`). | | ||
| | `ipv6` | string | No | Unbracketed IPv6 address (e.g., `"::1"`, `"2001:db8::1"`). | | ||
| | `port` | int | **Yes** | TCP port number (**must be 1–65535**). | | ||
|
|
||
| > **Notes**: | ||
| > - Service IDs must be non-empty strings. | ||
| > - **At least one** of `domain`, `ipv4`, or `ipv6` must be non-empty per instance. | ||
| > - Invalid or missing ports will cause initialization to fail. | ||
|
|
||
| ## Address Selection Logic | ||
|
|
||
| For each instance, the resolver selects the **first non-empty address** in this priority order: | ||
|
|
||
| 1. `domain` → e.g., `github.com` | ||
| 2. `ipv4` → e.g., `192.168.1.10` | ||
| 3. `ipv6` → e.g., `::1` | ||
|
|
||
| The final target address is formatted as: | ||
|
|
||
| - `host:port` for domain/IPv4 | ||
| - `[ipv6]:port` for IPv6 (automatically bracketed) | ||
|
|
||
| If a service has **multiple instances**, one is selected **uniformly at random** on each call. | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Inline JSON | ||
| ```yaml | ||
| configuration: | ||
| structuredType: "json" | ||
| stringValue: '{"appInstances":{"myapp":[{"ipv4":"127.0.0.1","port":4433}]}}' | ||
| ``` | ||
| → Resolves `"myapp"` to `127.0.0.1:4433` | ||
|
|
||
| ### Inline YAML (multi-line) | ||
| ```yaml | ||
| configuration: | ||
| structuredType: "yaml" | ||
| stringValue: | | ||
| appInstances: | ||
| myapp: | ||
| - domain: "example.com" | ||
| port: 80 | ||
| - ipv6: "::1" | ||
| port: 8080 | ||
| ``` | ||
| → Possible results: `example.com:80` or `[::1]:8080` (chosen randomly) | ||
|
|
||
| ### From External File | ||
| ```yaml | ||
| configuration: | ||
| structuredType: "yamlFile" | ||
| filePath: "/etc/dapr/services.yaml" | ||
| ``` | ||
|
|
||
| With `/etc/dapr/services.yaml`: | ||
| ```yaml | ||
| appInstances: | ||
| backend: | ||
| - ipv4: "10.0.0.5" | ||
| port: 3000 | ||
| ``` | ||
| → Resolves `"backend"` to `10.0.0.5:3000` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # yaml-language-server: $schema=../../component-metadata-schema.json | ||
| schemaVersion: v1 | ||
| type: nameresolution | ||
| name: structuredformat | ||
| version: v1 | ||
| status: alpha | ||
| title: "StructuredFormat" | ||
| urls: | ||
| - title: Reference | ||
| url: https://docs.dapr.io/reference/components-reference/supported-name-resolution/nr-structuredformat/ | ||
| metadata: | ||
| - name: structuredType | ||
| type: string | ||
| required: true | ||
| allowedValues: ["json", "yaml", "jsonFile", "yamlFile"] | ||
| description: Format type of the structured data. | ||
| example: "json" | ||
| - name: stringValue | ||
| type: string | ||
| required: false | ||
| description: Inline JSON/YAML string (required if structuredType is json/yaml). | ||
| example: '{"appInstances":{"myapp":[{"ipv4":"127.0.0.1","port":4433}]}}' | ||
| - name: filePath | ||
| type: string | ||
| required: false | ||
| description: Path to JSON/YAML file (required if structuredType is jsonFile/yamlFile). | ||
| example: "/etc/dapr/services.yaml" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,249 @@ | ||
| /* | ||
| Copyright 2025 The Dapr Authors | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package structuredformat | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "errors" | ||
| "fmt" | ||
| "math/rand" | ||
| "net" | ||
| "os" | ||
| "reflect" | ||
| "strconv" | ||
| "strings" | ||
| "time" | ||
|
|
||
| yaml "gopkg.in/yaml.v3" | ||
|
|
||
| "github.com/dapr/components-contrib/metadata" | ||
| nr "github.com/dapr/components-contrib/nameresolution" | ||
| "github.com/dapr/kit/logger" | ||
| kitmd "github.com/dapr/kit/metadata" | ||
| ) | ||
|
|
||
| const ( | ||
| JSONStructuredValue = "json" | ||
| YAMLStructuredValue = "yaml" | ||
| JSONFileStructuredValue = "jsonFile" | ||
| YAMLFileStructuredValue = "yamlFile" | ||
| ) | ||
|
|
||
| var allowedStructuredTypes = []string{ | ||
| JSONStructuredValue, | ||
| YAMLStructuredValue, | ||
| JSONFileStructuredValue, | ||
| YAMLFileStructuredValue, | ||
| } | ||
|
|
||
| // StructuredFormatResolver parses service names from a structured string | ||
| // defined in the configuration. | ||
| type StructuredFormatResolver struct { | ||
| meta structuredFormatMetadata | ||
| instances appInstances | ||
| logger logger.Logger | ||
| rand *rand.Rand | ||
| } | ||
|
|
||
| // structuredFormatMetadata represents the structured string (such as JSON or YAML) | ||
| // provided in the configuration for name resolution. | ||
| type structuredFormatMetadata struct { | ||
| StructuredType string `mapstructure:"structuredType"` | ||
| StringValue string `mapstructure:"stringValue"` | ||
| FilePath string `mapstructure:"filePath"` | ||
| } | ||
|
|
||
| // appInstances stores the relationship between services and their instances. | ||
| type appInstances struct { | ||
| AppInstances map[string][]address `json:"appInstances" yaml:"appInstances"` | ||
| } | ||
|
|
||
| // address contains service instance information. | ||
| type address struct { | ||
| Domain string `json:"domain" yaml:"domain"` | ||
| IPv4 string `json:"ipv4" yaml:"ipv4"` | ||
| IPv6 string `json:"ipv6" yaml:"ipv6"` | ||
| Port int `json:"port" yaml:"port"` | ||
| } | ||
|
|
||
| // isValid checks if the address has at least one valid host field. | ||
| func (a address) isValid() bool { | ||
| return (a.Domain != "" || a.IPv4 != "" || a.IPv6 != "") | ||
| } | ||
|
|
||
| // NewResolver creates a new Structured Format resolver. | ||
| func NewResolver(logger logger.Logger) nr.Resolver { | ||
| src := rand.NewSource(time.Now().UnixNano()) | ||
| return &StructuredFormatResolver{ | ||
| logger: logger, | ||
| // gosec is complaining that we are using a non-crypto-safe PRNG. | ||
| // This is fine in this scenario since we are using it only for selecting a random address for load-balancing. | ||
| //nolint:gosec | ||
| rand: rand.New(src), | ||
| } | ||
| } | ||
|
|
||
| // Init initializes the structured format resolver with the given metadata. | ||
| func (r *StructuredFormatResolver) Init(ctx context.Context, metadata nr.Metadata) error { | ||
| var meta structuredFormatMetadata | ||
| if err := kitmd.DecodeMetadata(metadata.Configuration, &meta); err != nil { | ||
| return fmt.Errorf("failed to decode metadata: %w", err) | ||
| } | ||
|
|
||
| // Validate structuredType | ||
| if !isValidStructuredType(meta.StructuredType) { | ||
| return fmt.Errorf("invalid structuredType %q; must be one of: %s", | ||
| meta.StructuredType, strings.Join(allowedStructuredTypes, ", ")) | ||
| } | ||
|
|
||
| // Validate required fields based on type | ||
| switch meta.StructuredType { | ||
| case JSONStructuredValue, YAMLStructuredValue: | ||
| if meta.StringValue == "" { | ||
| return fmt.Errorf("stringValue is required when structuredType is %q", meta.StructuredType) | ||
| } | ||
| case JSONFileStructuredValue, YAMLFileStructuredValue: | ||
| if meta.FilePath == "" { | ||
| return fmt.Errorf("filePath is required when structuredType is %q", meta.StructuredType) | ||
| } | ||
| } | ||
|
|
||
| r.meta = meta | ||
|
|
||
| instances, err := loadStructuredFormatData(r) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to load structured data: %w", err) | ||
| } | ||
|
|
||
| // validate that all addresses are valid | ||
| for serviceID, addrs := range instances.AppInstances { | ||
| for i, addr := range addrs { | ||
| if !addr.isValid() { | ||
| return fmt.Errorf("invalid address at AppInstances[%q][%d]: missing domain, ipv4, and ipv6", serviceID, i) | ||
| } | ||
| if addr.Port <= 0 || addr.Port > 65535 { | ||
| return fmt.Errorf("invalid port %d for AppInstances[%q][%d]", addr.Port, serviceID, i) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| r.instances = instances | ||
| return nil | ||
| } | ||
|
|
||
| // ResolveID resolves a service ID to an address using the configured value. | ||
| func (r *StructuredFormatResolver) ResolveID(ctx context.Context, req nr.ResolveRequest) (string, error) { | ||
| if req.ID == "" { | ||
| return "", errors.New("empty ID not allowed") | ||
| } | ||
|
|
||
| addresses, exists := r.instances.AppInstances[req.ID] | ||
| if !exists || len(addresses) == 0 { | ||
| return "", fmt.Errorf("no services found with ID %q", req.ID) | ||
| } | ||
|
|
||
| // Select a random instance (load balancing) | ||
| selected := addresses[r.rand.Intn(len(addresses))] | ||
|
|
||
| // Prefer Domain > IPv4 > IPv6 | ||
| host := selected.Domain | ||
| if host == "" { | ||
| host = selected.IPv4 | ||
| } | ||
| if host == "" { | ||
| host = selected.IPv6 | ||
| } | ||
|
|
||
| // This should not happen due to validation in Init, but be defensive. | ||
| if host == "" { | ||
| return "", fmt.Errorf("resolved address for %q has no valid host", req.ID) | ||
| } | ||
|
|
||
| return net.JoinHostPort(host, strconv.Itoa(selected.Port)), nil | ||
| } | ||
|
|
||
| // Close implements io.Closer. | ||
| func (r *StructuredFormatResolver) Close() error { | ||
| return nil | ||
| } | ||
|
|
||
| // GetComponentMetadata returns metadata info used for documentation and validation. | ||
| func (r *StructuredFormatResolver) GetComponentMetadata() metadata.MetadataMap { | ||
| m := metadata.MetadataMap{} | ||
| metadata.GetMetadataInfoFromStructType( | ||
| reflect.TypeOf(structuredFormatMetadata{}), | ||
| &m, | ||
| metadata.NameResolutionType, | ||
| ) | ||
| return m | ||
| } | ||
|
|
||
| // isValidStructuredType checks if the given type is allowed. | ||
| func isValidStructuredType(t string) bool { | ||
| for _, allowed := range allowedStructuredTypes { | ||
| if t == allowed { | ||
| return true | ||
| } | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| // loadStructuredFormatData loads the mapping from structured input. | ||
| func loadStructuredFormatData(r *StructuredFormatResolver) (appInstances, error) { | ||
| var instances appInstances | ||
|
|
||
| var data []byte | ||
| var err error | ||
|
|
||
| switch r.meta.StructuredType { | ||
| case JSONStructuredValue, YAMLStructuredValue: | ||
| data = []byte(r.meta.StringValue) | ||
| case JSONFileStructuredValue, YAMLFileStructuredValue: | ||
| // Security note: Consider restricting file access in production (e.g., allowlist paths). | ||
| data, err = os.ReadFile(r.meta.FilePath) | ||
| if err != nil { | ||
| return instances, fmt.Errorf("failed to read file %q: %w", r.meta.FilePath, err) | ||
| } | ||
| default: | ||
| // Should not happen due to prior validation | ||
| return instances, fmt.Errorf("unsupported structuredType: %s", r.meta.StructuredType) | ||
| } | ||
|
|
||
| // Parse based on format | ||
| switch r.meta.StructuredType { | ||
| case JSONStructuredValue, JSONFileStructuredValue: | ||
| err = json.Unmarshal(data, &instances) | ||
| case YAMLStructuredValue, YAMLFileStructuredValue: | ||
| err = yaml.Unmarshal(data, &instances) | ||
| } | ||
|
|
||
| if err != nil { | ||
| return instances, fmt.Errorf("failed to parse %s data: %w", getFormatName(r.meta.StructuredType), err) | ||
| } | ||
|
|
||
| return instances, nil | ||
| } | ||
|
|
||
| // getFormatName returns a human-readable format name. | ||
| func getFormatName(t string) string { | ||
| switch t { | ||
| case JSONStructuredValue, JSONFileStructuredValue: | ||
| return "JSON" | ||
| case YAMLStructuredValue, YAMLFileStructuredValue: | ||
| return "YAML" | ||
| default: | ||
| return t | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.