diff --git a/README.md b/README.md index d2af363..f766189 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,6 @@ This repository contains the official documentation for the DeployStack ecosystem. Visit [deploystack.io](https://deploystack.io) to learn more about our platform. -![GitHub Release](https://img.shields.io/github/v/release/deploystackio/documentation) -![GitHub deployments](https://img.shields.io/github/deployments/deploystackio/documentation/Production?label=Production) - ## Project Structure Key directories in this repository: diff --git a/docs/deploystack/deploystack-config-file.md b/docs/deploystack/deploystack-config-file.md index 1e5d029..733b264 100644 --- a/docs/deploystack/deploystack-config-file.md +++ b/docs/deploystack/deploystack-config-file.md @@ -43,9 +43,11 @@ You can override these values using the `config.yml` (only on your main branch) | Property | Type | Description | Constraints | |----------|------|-------------|-------------| -| `name` | String | Override the repository name for DeployStack display | Maximum 40 characters | -| `description` | String | Override the repository description for DeployStack display | Maximum 500 characters | -| `logo` | String | URL to your application logo | [Application Logo Configuration](/docs/deploystack/application-logo-configuration.md) | +| `mappings` | Array | Defines relationships between services for connection configuration | Required | +| `mappings[].fromService` | String | Service that needs to connect to another service | Required | +| `mappings[].toService` | String | Service being connected to | Required | +| `mappings[].environmentVariables` | Array of Strings | Environment variable names that reference the target service | Required | +| `mappings[].property` | String | Type of connection property to reference (e.g., 'connectionString', 'hostport') | Optional, defaults to 'hostport' | The override process follows this order: @@ -64,6 +66,7 @@ You can configure multiple branch deployments using the `deployment.branches` se | `description` | String | Explain the branch's purpose or version | Maximum 100 characters | | `active` | Boolean | Whether this branch is available for deployment | Optional, defaults to true | | `priority` | Number | Order in which branches appear (lower numbers first) | Minimum value: 1 | +| `exclude_providers` | Array | Optional list of cloud providers to exclude from template generation for this branch | Values must be valid provider codes: "aws", "rnd", "dop" | The default branch always has `priority: 0` and appears first in the deployment options, regardless of other branch priorities. @@ -80,6 +83,8 @@ deployment: label: "Beta (v2.x)" description: "Preview of upcoming v2.x release" priority: 1 + exclude_providers: + - "aws" # Exclude AWS CloudFormation for this branch v3: label: "Alpha (v3.x)" description: "Early preview of v3.x" @@ -102,6 +107,93 @@ When multiple branches are configured: This is especially useful for projects that maintain multiple active versions simultaneously, such as stable and beta releases. +The optional `exclude_providers` array allows you to specify which cloud providers should be excluded from template generation for particular branches. This is useful when certain features in a branch version may not be compatible with specific cloud providers. Valid provider codes are: + +Please check our [current supported provider list here](/docs/docker-to-iac/parser/index.md). + +For example, if your beta version uses features only supported in DigitalOcean, you might exclude the other providers: + +```yaml +v2-beta: + label: "Beta" + description: "Beta version with DigitalOcean-specific features" + exclude_providers: + - "aws" + - "rnd" +``` + +If no providers are excluded, templates will be generated for all supported cloud providers. + +### Service Connections + +You can configure service-to-service communication for multi-container applications using the `serviceConnections` property within each branch configuration. This feature is particularly useful for applications where services need to communicate with each other (e.g., web apps connecting to databases). + +| Property | Type | Description | Constraints | +|----------|------|-------------|-------------| +| `mappings` | Array | Defines relationships between services for connection configuration | Required | +| `mappings[].fromService` | String | Service that needs to connect to another service | Required | +| `mappings[].toService` | String | Service being connected to | Required | +| `mappings[].environmentVariables` | Array of Strings | Environment variable names that reference the target service | Required | + +Example configuration for service connections: + +```yaml +deployment: + branches: + main: + label: "Production" + description: "Production release" + serviceConnections: + mappings: + - fromService: "app" + toService: "db" + environmentVariables: + - "DATABASE_HOST" + - "DATABASE_URL" + property: "connectionString" + - fromService: "frontend" + toService: "api" + environmentVariables: + - "API_URL" + property: "hostport" +``` + +This configuration tells DeployStack how to properly configure communication between: + +- The "app" service and the "db" service through the DATABASE_HOST and DATABASE_URL environment variables +- The "frontend" service and the "api" service through the API_URL environment variable + +When templates are generated, DeployStack will transform these environment variables according to each cloud provider's specific service discovery mechanism: + +- For Render.com: Uses Blueprint's `fromService` syntax +- For DigitalOcean App Platform: Uses direct service name references + +For example, if your docker-compose.yml contains: + +```yaml +services: + app: + image: node:alpine + environment: + DATABASE_HOST: db + db: + image: mariadb:latest +``` + +The generated Render.com template would transform DATABASE_HOST to use their service discovery syntax: + +```yaml +services: + - name: app + # ...other configuration... + envVars: + - key: DATABASE_HOST + fromService: + name: db + type: pserv + property: hostport +``` + ## Schema Validation The configuration file is automatically validated against our JSON Schema when using supported IDEs (VS Code, IntelliJ, etc.). The schema is available at: @@ -127,6 +219,41 @@ application: logo: "https://example.com/logos/redis-manager.png" ``` +### Configure Multiple Branch Deployments with Service Connections + +```yaml +deployment: + branches: + stable: + label: "Stable" + description: "Production-ready version" + priority: 1 + serviceConnections: + mappings: + - fromService: "web" + toService: "api" + environmentVariables: + - "API_ENDPOINT" + - fromService: "api" + toService: "db" + environmentVariables: + - "DB_HOST" + - "DB_CONNECTION" + + beta: + label: "Beta" + description: "Preview of upcoming features" + priority: 2 + exclude_providers: + - "aws" # Beta version not supported on AWS + serviceConnections: + mappings: + - fromService: "web" + toService: "api" + environmentVariables: + - "API_ENDPOINT" +``` + ### Minimal Configuration example for logo update Please visit our [Application Logo Configuration](/docs/deploystack/application-logo-configuration.md) page. diff --git a/docs/docker-to-iac/api.md b/docs/docker-to-iac/api.md index 320be0c..be8282e 100644 --- a/docs/docker-to-iac/api.md +++ b/docs/docker-to-iac/api.md @@ -129,6 +129,7 @@ translate(input: string, options: { environmentVariableGeneration?: EnvironmentVariableGenerationConfig; environmentVariables?: Record; persistenceKey?: string; + serviceConnections?: ServiceConnectionsConfig; }): TranslationResult ``` @@ -139,6 +140,7 @@ interface TranslationResult { files: { [path: string]: FileOutput }; + serviceConnections?: ResolvedServiceConnection[]; } interface FileOutput { @@ -215,6 +217,45 @@ Object.entries(result.files).forEach(([path, fileData]) => { }); ``` +#### Configuring Service Connections + +```javascript +import { translate } from '@deploystack/docker-to-iac'; + +const dockerComposeContent = ` +version: "3" +services: + db: + image: mariadb:latest + environment: + - MYSQL_ROOT_PASSWORD=rootpass + app: + image: node:alpine + environment: + - DATABASE_HOST=db # This will be transformed +`; + +const result = translate(dockerComposeContent, { + source: 'compose', + target: 'DOP', // DigitalOcean App Platform + templateFormat: 'yaml', + serviceConnections: { + mappings: [ + { + fromService: 'app', // Service that needs to connect + toService: 'db', // Service to connect to + environmentVariables: [ // Env vars that reference the service + 'DATABASE_HOST' + ] + } + ] + } +}); + +// The result will include transformed service references: +console.log(result.serviceConnections); +``` + ### Example Output (AWS CloudFormation) ```yaml @@ -373,11 +414,73 @@ const result = translate(dockerConfig, { #### `options.persistenceKey?: string` -Optional. The `persistenceKey` parameter allows you to maintain consistent variable values across multiple template generations +Optional. The `persistenceKey` parameter allows you to maintain consistent variable values across multiple template generations. + +#### `options.serviceConnections?: ServiceConnectionsConfig` + +Optional. Configure service-to-service communications by defining which environment variables reference other services. + +```typescript +type ServiceConnectionsConfig = { + mappings: Array<{ + fromService: string; // Service that needs to connect + toService: string; // Service to connect to + environmentVariables: string[]; // Environment variables that reference the service + property?: string; // Connection property type (connectionString, hostport, etc.) + }> +}; +``` + +This option is currently supported by: + +- Render.com (RND): Uses Blueprint's `fromService` syntax +- DigitalOcean App Platform (DOP): Uses direct service names + +Example: + +```javascript +serviceConnections: { + mappings: [ + { + fromService: 'frontend', + toService: 'api', + environmentVariables: ['API_URL'], + property: 'hostport' + }, + { + fromService: 'app', + toService: 'db', + environmentVariables: ['DATABASE_URL'], + property: 'connectionString' + } + ] +} +``` ### Return Value -Returns the translated Infrastructure as Code template in the specified format. The structure and content will vary based on the target IaC language and template format chosen. +Returns the translated Infrastructure as Code template and any resolved service connections: + +```typescript +{ + files: { + // Generated IaC template files with paths as keys + 'render.yaml': { content: '...', format: 'yaml', isMain: true } + }, + serviceConnections: [ + { + fromService: 'app', + toService: 'db', + variables: { + 'DATABASE_HOST': { + originalValue: 'db', + transformedValue: 'db' // Transformed as appropriate for the provider + } + } + } + ] +} +``` ## List Services API diff --git a/docs/docker-to-iac/publishing-to-npm.md b/docs/docker-to-iac/publishing-to-npm.md index eada088..19fd727 100644 --- a/docs/docker-to-iac/publishing-to-npm.md +++ b/docs/docker-to-iac/publishing-to-npm.md @@ -5,7 +5,7 @@ menuTitle: Publishing to NPM # Publishing docker-to-iac module to NPM -We have created an organization @deploystack for NPM. Publishing in NPM happens automatically through `semantic-release`. Config: [https://github.com/deploystackio/docker-to-iac/blob/main/.github/workflows/release.yml](https://github.com/deploystackio/docker-to-iac/blob/main/.github/workflows/release.yml) +We have created an organization @deploystack for NPM. Publishing in NPM happens automatically through `semantic-release`. Config: [https://github.com/deploystackio/docker-to-iac/blob/main/.github/workflows/release-pr.yml](https://github.com/deploystackio/docker-to-iac/blob/main/.github/workflows/release-pr.yml) The prerequisite for a release is a successful pull request to the default branch `main`. This means that all tests must first be successfully completed. `semantic-release` creates a new version and automatically publishes the node package to: [https://www.npmjs.com/package/@deploystack/docker-to-iac](https://www.npmjs.com/package/@deploystack/docker-to-iac) diff --git a/docs/docker-to-iac/service-connections.md b/docs/docker-to-iac/service-connections.md new file mode 100644 index 0000000..118dba7 --- /dev/null +++ b/docs/docker-to-iac/service-connections.md @@ -0,0 +1,226 @@ +--- +description: Configure service-to-service communication in multi-container applications with docker-to-iac's service connections feature. Transform Docker Compose service references to cloud provider formats. +menuTitle: Service Connections +--- + +# Service Connections + +The `docker-to-iac` module supports configuring service-to-service communication when translating Docker Compose files to cloud provider IaC templates. This feature is essential for multi-container applications where services need to communicate with each other (e.g., web applications connecting to databases). + +## The Challenge + +In Docker Compose, services can communicate with each other using the service name as the hostname: + +```yaml +# docker-compose.yml +services: + db: + image: mariadb:latest + # ... + + app: + image: myapp:latest + environment: + - DATABASE_HOST=db # Reference to the db service + - API_URL=http://api:3000 # Reference within a URL + # ... +``` + +However, when deploying to cloud providers, each has its own format for service discovery: + +- **Render.com**: Uses Blueprint's `fromService` syntax for service references +- **DigitalOcean App Platform**: Services connect using the service name directly + +The Service Connections feature automatically configures these references based on the target cloud provider. + +## Supported Providers + +Currently, service connections are supported for: + +| Provider | Implementation | Example Reference | +|----------|---------------|------------------| +| Render.com | Blueprint's `fromService` | `fromService: { name: "db", type: "web", property: "hostport" }` | +| DigitalOcean App Platform | Direct service name | `db` or `http://api:3000` | + +> **Note**: AWS CloudFormation is not supported for service connections as it does not provide a direct way to reference services by name across tasks in the generated architecture. + +## Usage + +To configure service connections, use the `serviceConnections` option in the `translate` function: + +```javascript +import { translate } from '@deploystack/docker-to-iac'; + +const result = translate(dockerComposeContent, { + source: 'compose', + target: 'RND', // Or 'DOP' + templateFormat: 'yaml', + serviceConnections: { + mappings: [ + { + fromService: 'app', // Service that needs to connect + toService: 'db', // Service to connect to + environmentVariables: [ // List of env vars that reference the service + 'DATABASE_HOST', + 'API_URL' + ], + property: 'connectionString' // The type of connection (connectionString, hostport, etc.) + } + ] + } +}); +``` + +### Configuration Options + +For each service connection mapping: + +- `fromService`: The service that needs to connect to another service +- `toService`: The service being connected to +- `environmentVariables`: Array of environment variable names that reference the target service +- `property`: The type of connection property to reference (e.g., 'connectionString', 'hostport', etc.) + +## Provider-Specific Implementations + +### Render.com + +For Render.com, the module generates Blueprint configurations using the native `fromService` syntax: + +```yaml +# Generated Render Blueprint +services: + - name: app + # ...other configuration... + envVars: + # Regular environment variables + - key: NODE_ENV + value: production + + # Service reference using fromService + - key: DATABASE_HOST + fromService: + name: db + type: pserv + property: hostport # This property is derived from the 'property' parameter in your mapping +``` + +This approach leverages Render's built-in service discovery capabilities for reliable inter-service communication. + +### DigitalOcean App Platform + +For DigitalOcean App Platform, the module maintains the direct service name references, as services can communicate directly using the service name: + +```yaml +# Generated DigitalOcean App Spec +services: + - name: app + # ...other configuration... + envs: + - key: DATABASE_HOST + value: db + - key: API_URL + value: http://api:3000 +``` + +## Complete Example + +Here's a complete example showing Node.js microservices communicating with each other, and a more detailed database connection example: + +```javascript +const dockerComposeContent = ` +version: "3" + +services: + api: + image: node:18-alpine + command: node /app/server.js + ports: + - "3000:3000" + + frontend: + image: node:18-alpine + environment: + - API_URL=http://api:3000 # This will be transformed appropriately + ports: + - "8080:8080" +`; + +const serviceConnections = { + mappings: [ + { + fromService: 'frontend', + toService: 'api', + environmentVariables: ['API_URL'], + property: 'hostport' + } + ] +}; + +// For DigitalOcean - Service name stays as "api" in http://api:3000 +// For Render - Will use fromService syntax instead of string replacement + +// Database Connection Example +const databaseComposeContent = ` +services: + app: + image: node:18-alpine + environment: + - DATABASE_URL=postgresql://postgres:secret@postgres:5432/mydb + ports: + - "3000:3000" + + postgres: + image: postgres:latest + environment: + - POSTGRES_DB: mydb + - POSTGRES_USER: postgres + - POSTGRES_PASSWORD: secret +`; + +const dbServiceConnections = { + mappings: [ + { + fromService: 'app', + toService: 'postgres', + environmentVariables: ['DATABASE_URL'], + property: 'connectionString' // Use connectionString for full database URL + } + ] +}; + +// Result will include proper connection format for each provider +// DigitalOcean: DATABASE_URL=${postgres.DATABASE_URL} +// Render: fromDatabase: { name: "postgres", property: "connectionString" } +``` + +## Response Format + +The `translate` function returns information about the resolved service connections: + +```javascript +{ + files: { + // Generated IaC template files + }, + serviceConnections: [ + { + fromService: 'app', + toService: 'db', + variables: { + 'DATABASE_HOST': { + originalValue: 'db', + transformedValue: 'db' // For DigitalOcean or Render + } + } + } + ] +} +``` + +This information can be useful for debugging or understanding how service connections were processed. + +## Limitations + +- The feature only transforms environment variable values that exactly match the service name +- More complex connection strings must be handled separately +- The feature doesn't adjust ports or protocols, only service hostnames