From c34ee8245fd7d3a215c95864a50ae9884c91f1b6 Mon Sep 17 00:00:00 2001 From: Arjun Mehta Date: Tue, 12 May 2026 09:00:00 +0000 Subject: [PATCH] docs: add IDP architecture overview, entity design, and meeting notes - IDP architecture overview with Backstage component diagram - Service catalog entity design with ER diagram - Platform team standup notes (May 5) - Blueprint review meeting notes (May 12) Signed-off-by: Arjun Mehta --- docs/architecture/idp-overview.md | 83 ++++++++++++ docs/architecture/service-catalog-entities.md | 122 ++++++++++++++++++ .../2026-05-05-platform-team-standup.md | 34 +++++ docs/meetings/2026-05-12-blueprint-review.md | 48 +++++++ 4 files changed, 287 insertions(+) create mode 100644 docs/architecture/idp-overview.md create mode 100644 docs/architecture/service-catalog-entities.md create mode 100644 docs/meetings/2026-05-05-platform-team-standup.md create mode 100644 docs/meetings/2026-05-12-blueprint-review.md diff --git a/docs/architecture/idp-overview.md b/docs/architecture/idp-overview.md new file mode 100644 index 0000000..bc24ab8 --- /dev/null +++ b/docs/architecture/idp-overview.md @@ -0,0 +1,83 @@ +# Internal Developer Platform - Architecture Overview + +The TCS IDP is a Backstage-based developer portal delivering service catalog, golden paths (scaffolder templates), TechDocs, and developer workflow integrations to engineering teams. It provides a single pane of glass for developers to discover services, create new ones from standardized templates, and access operational information without context-switching across tools. + +## Architecture Diagram + +```mermaid +graph LR + Developer([Developer]) + + subgraph Backstage Portal + Catalog[Software Catalog] + Scaffolder[Scaffolder] + TechDocs[TechDocs] + K8sPlugin[Kubernetes Plugin] + GHActionsPlugin[GitHub Actions Plugin] + end + + subgraph GitHub + Repos[Repositories] + Actions[GitHub Actions] + GoldenPaths[Golden Path Templates] + end + + subgraph AWS + EKS[EKS Cluster] + S3[S3 TechDocs Bucket] + end + + Developer --> Catalog + Developer --> Scaffolder + Developer --> TechDocs + Developer --> K8sPlugin + Developer --> GHActionsPlugin + + GoldenPaths --> Scaffolder + Scaffolder -->|New Repo Generated| Repos + GHActionsPlugin --> Actions + K8sPlugin --> EKS + TechDocs -->|Read docs| S3 + Actions -->|Publish docs| S3 + Repos --> Catalog +``` + +## Core Capabilities + +### Software Catalog + +The software catalog is the central registry for all services, APIs, resources, and teams. Every deployed service must have a `catalog-info.yaml` in its repository. The catalog ingests this file automatically via GitHub discovery, providing a live view of the software ecosystem. + +### Scaffolder (Golden Paths) + +The scaffolder provides parameterized templates that generate new repositories pre-configured with CI/CD, Helm charts, TechDocs, and catalog registration. Teams use this instead of copying from example repos. Available golden paths: + +- Node.js microservice (Express + GitHub Actions + Helm) +- Python microservice (FastAPI + GitHub Actions + Helm) + +### TechDocs + +TechDocs turns Markdown documentation committed alongside code into rendered, searchable HTML. Documentation is built by GitHub Actions (`techdocs-cli generate` + `techdocs-cli publish`) and stored in S3. Backstage reads from S3 at browse time. No documentation system separate from the repo is needed. + +### Kubernetes Plugin + +Shows live cluster state (deployments, pods, resource usage) directly on the catalog service page. Developers see their service's Kubernetes health without leaving Backstage. Configured to read from the TCS shared EKS cluster. + +### GitHub Actions Plugin + +Shows CI/CD pipeline status for the service's GitHub Actions workflows directly on the catalog page. Surfaces build and deploy status without navigating to GitHub. + +## Integration Points + +| Integration | Purpose | Status | +|-------------|---------|--------| +| GitHub | Source of truth for code and CI/CD | Configured | +| EKS | Kubernetes cluster visibility | Configured | +| AWS Cost | Cost attribution per service | Planned | +| PagerDuty | On-call integration | Optional | + +## Getting Started + +To create a new service using the golden path templates, see the [tcs-platform-blueprints](https://github.com/tata-consulting/tcs-platform-blueprints) repository. Templates are available for Node.js and Python services. + +For catalog onboarding of existing services, see [Service Catalog Entity Design](./service-catalog-entities.md). diff --git a/docs/architecture/service-catalog-entities.md b/docs/architecture/service-catalog-entities.md new file mode 100644 index 0000000..c5cfc1f --- /dev/null +++ b/docs/architecture/service-catalog-entities.md @@ -0,0 +1,122 @@ +# Service Catalog Entity Design + +This document defines the entity kinds, annotation standards, and naming conventions for the TCS Backstage service catalog. + +## Entity Relationship Diagram + +```mermaid +erDiagram + Domain ||--o{ System : contains + System ||--o{ Component : contains + System ||--o{ Resource : contains + Component ||--o{ API : provides + Component ||--o{ API : consumes + Component ||--o{ Resource : depends-on + Group ||--o{ Component : owns + Group ||--o{ API : owns + Group ||--o{ Resource : owns + User }o--|| Group : member-of +``` + +## Entity Kinds + +### Component + +Deployable services and libraries. This is the most common entity kind. Every microservice, frontend application, data pipeline, or shared library should have a `Component` entry. + +**Subtypes (`spec.type`):** `service`, `library`, `website`, `pipeline` + +### API + +Machine-readable API specifications. Links to or embeds the API contract so consumers can discover and understand the interface without reading source code. + +**Subtypes (`spec.type`):** `openapi`, `asyncapi`, `grpc`, `graphql` + +API entities should reference the spec file via annotation rather than embedding the spec inline - see Required Annotations below. + +### Resource + +Managed infrastructure that services depend on. Represents external resources like databases, queues, and object storage that are not themselves deployable services. + +**Subtypes (`spec.type`):** `database`, `s3-bucket`, `message-queue`, `cache` + +### System + +A logical grouping of related Components, APIs, and Resources that together form a product or capability. Systems provide a higher-level view than individual components. + +### Domain + +Business domain grouping for Systems. Maps to organizational or product domain boundaries. + +### User / Group + +Team and individual identity. Used for ownership and on-call assignment. Typically synchronized from the identity provider rather than managed manually. + +## Required Annotations + +All `Component` entities must include the following annotations. The golden path scaffolder templates pre-populate these automatically. + +```yaml +apiVersion: backstage.io/v1alpha1 +kind: Component +metadata: + name: my-service # kebab-case, matches repo name + description: "Short description of the service" + annotations: + # Source control - enables GitHub integration + github.com/project-slug: tata-consulting/my-service + + # TechDocs - points to the docs directory + backstage.io/techdocs-ref: dir:. + + # TCS custom annotations (all required) + tcs.io/team: platform # owning team slug + tcs.io/tier: tier-2 # tier-1 | tier-2 | tier-3 + tcs.io/language: nodejs # nodejs | python | java | go | etc. + tcs.io/lifecycle: production # production | development | deprecated | experimental + tcs.io/maturity: beta # experimental | alpha | beta | ga | deprecated + + # Optional but recommended + tcs.io/oncall: my-pd-schedule-id # PagerDuty schedule ID or OpsGenie team slug +spec: + type: service + lifecycle: production + owner: group:platform +``` + +### API Entity Annotation + +For `API` entities, reference the spec file rather than embedding it: + +```yaml +apiVersion: backstage.io/v1alpha1 +kind: API +metadata: + name: my-service-api + annotations: + backstage.io/definition-at: https://raw.githubusercontent.com/tata-consulting/my-service/main/openapi.yaml +spec: + type: openapi + lifecycle: production + owner: group:platform + definition: "" # populated by definition-at at runtime +``` + +## Naming Conventions + +| Field | Convention | Example | +|-------|-----------|---------| +| `metadata.name` | `kebab-case`, match the GitHub repo name | `payments-service` | +| `tcs.io/team` | `kebab-case` team slug | `platform-team` | +| `tcs.io/tier` | `tier-1`, `tier-2`, or `tier-3` | `tier-1` | +| System names | `-` kebab-case | `fintech-payments` | +| Domain names | single word or short kebab | `fintech` | +| Group names | mirror GitHub team names | `platform-team` | + +## Tier Definitions + +| Tier | SLO Required | On-Call Required | Example | +|------|-------------|------------------|---------| +| `tier-1` | Yes | Yes | Payment processing, auth service | +| `tier-2` | Recommended | No | Internal tooling, reporting services | +| `tier-3` | No | No | Deprecated services pending retirement | diff --git a/docs/meetings/2026-05-05-platform-team-standup.md b/docs/meetings/2026-05-05-platform-team-standup.md new file mode 100644 index 0000000..5ad2906 --- /dev/null +++ b/docs/meetings/2026-05-05-platform-team-standup.md @@ -0,0 +1,34 @@ +# Platform Team Standup - Sprint Review +**Date:** 2026-05-05 +**Attendees:** Arjun Mehta, Saraj Krishna Singh, Mia Cycle, Pontus Ringblom, Winkletinkle +**Sprint:** Platform IDP - Sprint 2 + +## Status Updates + +**Saraj (Node.js golden path template):** +First draft of scaffolder template complete. Generates: Express app skeleton, Dockerfile (multi-stage, alpine), GitHub Actions CI workflow, catalog-info.yaml with mandatory annotations, basic Helm chart. Needs review on Helm chart resource limits. + +**Pontus (Python golden path template):** +In progress. FastAPI skeleton done, Dockerfile done. CI workflow and MkDocs setup remaining. ETA: end of week. + +**Mia (TechDocs pipeline):** +S3 bucket provisioned in dev. Reusable GitHub Actions workflow drafted. Testing with the platform repo's own TechDocs - finding some issues with MkDocs plugin version compatibility. + +**Winkletinkle (Service catalog population):** +Started populating catalog with first 5 internal services. Hitting annotation inconsistency - teams have slightly different formats. Proposing a catalog linting step in CI. + +**Arjun (Backstage app-config):** +App-config for dev environment is deployed. GitHub OAuth working. EKS plugin configured and showing cluster data. Proceeding to production app-config. + +## Blockers +- MkDocs plugin version compatibility (Mia) - investigating, ETA tomorrow +- Service catalog annotation inconsistency (Winkletinkle) - proposing lint step PR + +## Action Items +| Owner | Action | Due | +|-------|--------|-----| +| Saraj | Request review on Node.js template PR | 2026-05-06 | +| Pontus | Complete Python template | 2026-05-09 | +| Mia | Resolve MkDocs version issue | 2026-05-06 | +| Winkletinkle | PR for catalog-info linting CI step | 2026-05-07 | +| Arjun | Draft production app-config | 2026-05-08 | diff --git a/docs/meetings/2026-05-12-blueprint-review.md b/docs/meetings/2026-05-12-blueprint-review.md new file mode 100644 index 0000000..c876ed9 --- /dev/null +++ b/docs/meetings/2026-05-12-blueprint-review.md @@ -0,0 +1,48 @@ +# Platform Blueprint Review +**Date:** 2026-05-12 +**Attendees:** Arjun Mehta, Saraj Krishna Singh, Mia Cycle, Pontus Ringblom, Winkletinkle +**Type:** Blueprint design review + +## Agenda +1. Node.js golden path - review and sign-off +2. Python golden path - review and sign-off +3. Microservices blueprint - review patterns and standards +4. Blueprint maturity model + +## Discussion + +### Node.js Golden Path +Reviewed. Approved with minor changes: +- Add `tier` input defaulting to `tier-2` +- Add checkov scan step to CI workflow +- Helm chart: add HPA and proper resource requests/limits +Saraj to apply changes, then merge. + +### Python Golden Path +Reviewed. Approved with same changes as Node.js plus: +- Switch base image from `python:3.12-slim` to `python:3.12-alpine` for smaller images +- Add `ruff` configuration in `pyproject.toml` +Pontus to apply, then merge. + +### Microservices Blueprint +Draft reviewed. Key decisions: +- mTLS: "service mesh optional, mTLS via cert-manager if no mesh" (not prescriptive on Istio) +- Observability: OpenTelemetry SDK standard, export target is client-specific +- gRPC: deferred to v2 blueprint +Arjun to finalize and merge. + +### Blueprint Maturity Model +Approved. Maturity annotation (`tcs.io/maturity`) and sunset date (`tcs.io/sunset-date` for deprecated) added to standards. Both golden paths will launch as `beta`. + +## Decisions +1. Node.js and Python golden paths approved (with noted changes), launch as Beta +2. Microservices blueprint: mTLS pattern updated, launch as Beta +3. Blueprint maturity model: adopted + +## Action Items +| Owner | Action | Due | +|-------|--------|-----| +| Saraj | Apply Node.js template feedback, merge | 2026-05-13 | +| Pontus | Apply Python template feedback, merge | 2026-05-14 | +| Arjun | Finalize microservices blueprint | 2026-05-15 | +| Mia | Add maturity annotations to all merged blueprints | 2026-05-15 |