Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions docs/architecture/idp-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Internal Developer Platform - Architecture Overview

The TCS IDP is a Backstage-based developer portal delivering service catalog, golden paths (scaffolder templates), TechDocs, and developer workflow integrations to engineering teams. It provides a single pane of glass for developers to discover services, create new ones from standardized templates, and access operational information without context-switching across tools.

## Architecture Diagram

```mermaid
graph LR
Developer([Developer])

subgraph Backstage Portal
Catalog[Software Catalog]
Scaffolder[Scaffolder]
TechDocs[TechDocs]
K8sPlugin[Kubernetes Plugin]
GHActionsPlugin[GitHub Actions Plugin]
end

subgraph GitHub
Repos[Repositories]
Actions[GitHub Actions]
GoldenPaths[Golden Path Templates]
end

subgraph AWS
EKS[EKS Cluster]
S3[S3 TechDocs Bucket]
end

Developer --> Catalog
Developer --> Scaffolder
Developer --> TechDocs
Developer --> K8sPlugin
Developer --> GHActionsPlugin

GoldenPaths --> Scaffolder
Scaffolder -->|New Repo Generated| Repos
GHActionsPlugin --> Actions
K8sPlugin --> EKS
TechDocs -->|Read docs| S3
Actions -->|Publish docs| S3
Repos --> Catalog
```

## Core Capabilities

### Software Catalog

The software catalog is the central registry for all services, APIs, resources, and teams. Every deployed service must have a `catalog-info.yaml` in its repository. The catalog ingests this file automatically via GitHub discovery, providing a live view of the software ecosystem.

### Scaffolder (Golden Paths)

The scaffolder provides parameterized templates that generate new repositories pre-configured with CI/CD, Helm charts, TechDocs, and catalog registration. Teams use this instead of copying from example repos. Available golden paths:

- Node.js microservice (Express + GitHub Actions + Helm)
- Python microservice (FastAPI + GitHub Actions + Helm)

### TechDocs

TechDocs turns Markdown documentation committed alongside code into rendered, searchable HTML. Documentation is built by GitHub Actions (`techdocs-cli generate` + `techdocs-cli publish`) and stored in S3. Backstage reads from S3 at browse time. No documentation system separate from the repo is needed.

### Kubernetes Plugin

Shows live cluster state (deployments, pods, resource usage) directly on the catalog service page. Developers see their service's Kubernetes health without leaving Backstage. Configured to read from the TCS shared EKS cluster.

### GitHub Actions Plugin

Shows CI/CD pipeline status for the service's GitHub Actions workflows directly on the catalog page. Surfaces build and deploy status without navigating to GitHub.

## Integration Points

| Integration | Purpose | Status |
|-------------|---------|--------|
| GitHub | Source of truth for code and CI/CD | Configured |
| EKS | Kubernetes cluster visibility | Configured |
| AWS Cost | Cost attribution per service | Planned |
| PagerDuty | On-call integration | Optional |

## Getting Started

To create a new service using the golden path templates, see the [tcs-platform-blueprints](https://github.com/tata-consulting/tcs-platform-blueprints) repository. Templates are available for Node.js and Python services.

For catalog onboarding of existing services, see [Service Catalog Entity Design](./service-catalog-entities.md).
122 changes: 122 additions & 0 deletions docs/architecture/service-catalog-entities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Service Catalog Entity Design

This document defines the entity kinds, annotation standards, and naming conventions for the TCS Backstage service catalog.

## Entity Relationship Diagram

```mermaid
erDiagram
Domain ||--o{ System : contains
System ||--o{ Component : contains
System ||--o{ Resource : contains
Component ||--o{ API : provides
Component ||--o{ API : consumes
Component ||--o{ Resource : depends-on
Group ||--o{ Component : owns
Group ||--o{ API : owns
Group ||--o{ Resource : owns
User }o--|| Group : member-of
```

## Entity Kinds

### Component

Deployable services and libraries. This is the most common entity kind. Every microservice, frontend application, data pipeline, or shared library should have a `Component` entry.

**Subtypes (`spec.type`):** `service`, `library`, `website`, `pipeline`

### API

Machine-readable API specifications. Links to or embeds the API contract so consumers can discover and understand the interface without reading source code.

**Subtypes (`spec.type`):** `openapi`, `asyncapi`, `grpc`, `graphql`

API entities should reference the spec file via annotation rather than embedding the spec inline - see Required Annotations below.

### Resource

Managed infrastructure that services depend on. Represents external resources like databases, queues, and object storage that are not themselves deployable services.

**Subtypes (`spec.type`):** `database`, `s3-bucket`, `message-queue`, `cache`

### System

A logical grouping of related Components, APIs, and Resources that together form a product or capability. Systems provide a higher-level view than individual components.

### Domain

Business domain grouping for Systems. Maps to organizational or product domain boundaries.

### User / Group

Team and individual identity. Used for ownership and on-call assignment. Typically synchronized from the identity provider rather than managed manually.

## Required Annotations

All `Component` entities must include the following annotations. The golden path scaffolder templates pre-populate these automatically.

```yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: my-service # kebab-case, matches repo name
description: "Short description of the service"
annotations:
# Source control - enables GitHub integration
github.com/project-slug: tata-consulting/my-service

# TechDocs - points to the docs directory
backstage.io/techdocs-ref: dir:.

# TCS custom annotations (all required)
tcs.io/team: platform # owning team slug
tcs.io/tier: tier-2 # tier-1 | tier-2 | tier-3
tcs.io/language: nodejs # nodejs | python | java | go | etc.
tcs.io/lifecycle: production # production | development | deprecated | experimental
tcs.io/maturity: beta # experimental | alpha | beta | ga | deprecated

# Optional but recommended
tcs.io/oncall: my-pd-schedule-id # PagerDuty schedule ID or OpsGenie team slug
spec:
type: service
lifecycle: production
owner: group:platform
```

### API Entity Annotation

For `API` entities, reference the spec file rather than embedding it:

```yaml
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
name: my-service-api
annotations:
backstage.io/definition-at: https://raw.githubusercontent.com/tata-consulting/my-service/main/openapi.yaml
spec:
type: openapi
lifecycle: production
owner: group:platform
definition: "" # populated by definition-at at runtime
```

## Naming Conventions

| Field | Convention | Example |
|-------|-----------|---------|
| `metadata.name` | `kebab-case`, match the GitHub repo name | `payments-service` |
| `tcs.io/team` | `kebab-case` team slug | `platform-team` |
| `tcs.io/tier` | `tier-1`, `tier-2`, or `tier-3` | `tier-1` |
| System names | `<domain>-<product>` kebab-case | `fintech-payments` |
| Domain names | single word or short kebab | `fintech` |
| Group names | mirror GitHub team names | `platform-team` |

## Tier Definitions

| Tier | SLO Required | On-Call Required | Example |
|------|-------------|------------------|---------|
| `tier-1` | Yes | Yes | Payment processing, auth service |
| `tier-2` | Recommended | No | Internal tooling, reporting services |
| `tier-3` | No | No | Deprecated services pending retirement |
34 changes: 34 additions & 0 deletions docs/meetings/2026-05-05-platform-team-standup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Platform Team Standup - Sprint Review
**Date:** 2026-05-05
**Attendees:** Arjun Mehta, Saraj Krishna Singh, Mia Cycle, Pontus Ringblom, Winkletinkle
**Sprint:** Platform IDP - Sprint 2

## Status Updates

**Saraj (Node.js golden path template):**
First draft of scaffolder template complete. Generates: Express app skeleton, Dockerfile (multi-stage, alpine), GitHub Actions CI workflow, catalog-info.yaml with mandatory annotations, basic Helm chart. Needs review on Helm chart resource limits.

**Pontus (Python golden path template):**
In progress. FastAPI skeleton done, Dockerfile done. CI workflow and MkDocs setup remaining. ETA: end of week.

**Mia (TechDocs pipeline):**
S3 bucket provisioned in dev. Reusable GitHub Actions workflow drafted. Testing with the platform repo's own TechDocs - finding some issues with MkDocs plugin version compatibility.

**Winkletinkle (Service catalog population):**
Started populating catalog with first 5 internal services. Hitting annotation inconsistency - teams have slightly different formats. Proposing a catalog linting step in CI.

**Arjun (Backstage app-config):**
App-config for dev environment is deployed. GitHub OAuth working. EKS plugin configured and showing cluster data. Proceeding to production app-config.

## Blockers
- MkDocs plugin version compatibility (Mia) - investigating, ETA tomorrow
- Service catalog annotation inconsistency (Winkletinkle) - proposing lint step PR

## Action Items
| Owner | Action | Due |
|-------|--------|-----|
| Saraj | Request review on Node.js template PR | 2026-05-06 |
| Pontus | Complete Python template | 2026-05-09 |
| Mia | Resolve MkDocs version issue | 2026-05-06 |
| Winkletinkle | PR for catalog-info linting CI step | 2026-05-07 |
| Arjun | Draft production app-config | 2026-05-08 |
48 changes: 48 additions & 0 deletions docs/meetings/2026-05-12-blueprint-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Platform Blueprint Review
**Date:** 2026-05-12
**Attendees:** Arjun Mehta, Saraj Krishna Singh, Mia Cycle, Pontus Ringblom, Winkletinkle
**Type:** Blueprint design review

## Agenda
1. Node.js golden path - review and sign-off
2. Python golden path - review and sign-off
3. Microservices blueprint - review patterns and standards
4. Blueprint maturity model

## Discussion

### Node.js Golden Path
Reviewed. Approved with minor changes:
- Add `tier` input defaulting to `tier-2`
- Add checkov scan step to CI workflow
- Helm chart: add HPA and proper resource requests/limits
Saraj to apply changes, then merge.

### Python Golden Path
Reviewed. Approved with same changes as Node.js plus:
- Switch base image from `python:3.12-slim` to `python:3.12-alpine` for smaller images
- Add `ruff` configuration in `pyproject.toml`
Pontus to apply, then merge.

### Microservices Blueprint
Draft reviewed. Key decisions:
- mTLS: "service mesh optional, mTLS via cert-manager if no mesh" (not prescriptive on Istio)
- Observability: OpenTelemetry SDK standard, export target is client-specific
- gRPC: deferred to v2 blueprint
Arjun to finalize and merge.

### Blueprint Maturity Model
Approved. Maturity annotation (`tcs.io/maturity`) and sunset date (`tcs.io/sunset-date` for deprecated) added to standards. Both golden paths will launch as `beta`.

## Decisions
1. Node.js and Python golden paths approved (with noted changes), launch as Beta
2. Microservices blueprint: mTLS pattern updated, launch as Beta
3. Blueprint maturity model: adopted

## Action Items
| Owner | Action | Due |
|-------|--------|-----|
| Saraj | Apply Node.js template feedback, merge | 2026-05-13 |
| Pontus | Apply Python template feedback, merge | 2026-05-14 |
| Arjun | Finalize microservices blueprint | 2026-05-15 |
| Mia | Add maturity annotations to all merged blueprints | 2026-05-15 |
Loading