diff --git a/README.md b/README.md index 3bf956197f..8d7b7a5abc 100644 --- a/README.md +++ b/README.md @@ -72,6 +72,12 @@ After configuring OpenC3 COSMOS to talk to your hardware, you can immediately us OpenC3 COSMOS was originally created by Ryan Melton (ryanmelt) and Jason Thomas (jmthomas) and is built and maintained by OpenC3, Inc. +## Dependencies + +### Taskfile + +This project uses [Taskfile](https://taskfile.dev) for task automation. See the [Taskfile installation guide](https://taskfile.dev/installation/) to install, then run `task` to see available commands. + ## Getting Started 1. See the [Installation Guide](https://docs.openc3.com/docs/getting-started/installation) for detailed instructions. diff --git a/Taskfile.yaml b/Taskfile.yaml new file mode 100644 index 0000000000..895000b2d0 --- /dev/null +++ b/Taskfile.yaml @@ -0,0 +1,708 @@ + +version: "3" + +vars: + CONTAINER_BIN: '{{.CONTAINER_BIN | default "docker"}}' + ENV_FILE: '{{.ENV_FILE | default ".env"}}' + COMPOSE_FILE: compose.yaml + COMPOSE_BUILD_FILE: compose-build.yaml + PORT: '{{.PORT | default "2900"}}' + # ANSI color codes for terminal output + YELLOW_BOLD: '\033[1;33m' + CYAN_BOLD: '\033[1;36m' + RESET: '\033[0m' + +tasks: + # ============================================================================ + # Common Commands + # ============================================================================ + + default: + desc: Show available tasks + silent: true + cmds: + - task --list + + start: + desc: Build and run COSMOS (development) or just run (runtime) + summary: | + Build and run OpenC3 COSMOS containers. + + In development environments (with compose-build.yaml), this will: + 1. Build all Docker containers from source + 2. Start the containers + + In runtime environments, this will just start the containers. + + Access COSMOS at: http://localhost:{{.PORT}} + silent: true + cmds: + - task: _check-root + - | + if test -f compose-build.yaml; then + task build + task run + else + task run + fi + - | + echo "" + echo "==========================================" + echo " OpenC3 COSMOS is now running!" + echo "==========================================" + echo "" + echo " Web Interface: http://localhost:{{.PORT}}" + echo "" + echo " Useful commands:" + echo " task ps - Show container status" + echo " task logs - Follow container logs" + echo " task stop - Stop all containers" + echo "" + + run: + desc: Start COSMOS containers in detached mode + summary: | + Start all OpenC3 COSMOS containers in detached mode. + + Containers will run in the background. Access COSMOS at http://localhost:{{.PORT}} + + Common services: + - openc3-operator - Main orchestration service + - openc3-cosmos-cmd-tlm-api - Command/Telemetry API + - openc3-cosmos-script-runner-api - Script execution service + - openc3-redis - Redis database + - openc3-buckets - Object storage + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - task: _check-root + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} up -d" + - | + echo "" + echo -e "{{.YELLOW_BOLD}}COSMOS is started. Access the web interface at:{{.RESET}}" + echo -e " {{.CYAN_BOLD}}http://localhost:{{.PORT}}{{.RESET}}" + echo "" + + stop: + desc: Stop all COSMOS containers gracefully + summary: | + Stop all OpenC3 COSMOS containers gracefully. + + This command: + 1. Stops operator, script-runner-api, and cmd-tlm-api containers + 2. Waits 5 seconds for graceful shutdown + 3. Runs docker compose down with 30 second timeout + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - echo "Stopping COSMOS containers..." + - echo " Stopping openc3-operator (if running)..." + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} stop openc3-operator" + - echo " Stopping openc3-cosmos-script-runner-api (if running)..." + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} stop openc3-cosmos-script-runner-api" + - echo " Stopping openc3-cosmos-cmd-tlm-api (if running)..." + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} stop openc3-cosmos-cmd-tlm-api" + - echo " Waiting 5 seconds for graceful shutdown..." + - sleep 5 + - echo " Running docker compose down..." + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} down -t 30" + - echo "COSMOS containers stopped." + + logs: + desc: Follow logs from all containers + summary: | + Follow logs from all COSMOS containers. + + Use Ctrl+C to stop following logs. + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} logs -f {{.CLI_ARGS}}" + + ps: + desc: Show status of COSMOS containers + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} ps" + + # ============================================================================ + # Development Commands + # ============================================================================ + + build: + desc: Build all COSMOS Docker containers from source + summary: | + Build all OpenC3 COSMOS Docker containers from source. + + This command: + 1. Runs setup to download certificates + 2. Builds openc3-ruby base image + 3. Builds openc3-base image + 4. Builds openc3-node image + 5. Builds all remaining service containers + + Requires compose-build.yaml (development environment). + dotenv: + - "{{.ENV_FILE}}" + preconditions: + - sh: test -f compose-build.yaml + msg: "Error: 'build' command is only available in development environments (compose-build.yaml not found)" + cmds: + - task: setup + # Handle restrictive umasks - Built files need to be world readable + - chmod -R +r . + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build openc3-ruby" + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build openc3-base" + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build openc3-node" + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build" + + build-image: + desc: Build a specific Docker image + summary: | + Build a specific OpenC3 COSMOS Docker image. + + Usage: task build-image -- + + Example: task build-image -- openc3-cosmos-cmd-tlm-api + dotenv: + - "{{.ENV_FILE}}" + preconditions: + - sh: test -f compose-build.yaml + msg: "Error: 'build-image' command is only available in development environments" + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build {{.CLI_ARGS}}" + + setup: + desc: Run initial setup (download certificates, etc.) + summary: | + Run initial setup for OpenC3 COSMOS. + + This downloads necessary certificates and performs other setup tasks. + cmds: + - ./scripts/linux/openc3_setup.sh + + # ============================================================================ + # UBI (Universal Base Image) Commands + # ============================================================================ + + start-ubi: + desc: Build and run COSMOS with UBI images + summary: | + Build and run OpenC3 COSMOS with Red Hat UBI (Universal Base Image) containers. + + Used for enterprise deployments requiring Red Hat UBI base images, + suitable for air-gapped and government environments. + cmds: + - task: _check-root + - | + if test -f compose-build.yaml; then + task build-ubi + task run-ubi + else + task run-ubi + fi + + build-ubi: + desc: Build COSMOS UBI containers + summary: | + Build OpenC3 COSMOS UBI (Universal Base Image) containers. + + Used for enterprise deployments requiring Red Hat UBI base images. + + Required environment variables (set in .env file): + OPENC3_UBI_REGISTRY - UBI registry URL + OPENC3_UBI_IMAGE - UBI image name + OPENC3_UBI_TAG - UBI image tag + dotenv: + - "{{.ENV_FILE}}" + preconditions: + - sh: test -f compose-build.yaml + msg: "Error: 'build-ubi' command is only available in development environments" + cmds: + - | + if test -f /etc/ssl/certs/ca-bundle.crt; then + cp /etc/ssl/certs/ca-bundle.crt cacert.pem + fi + - task: setup + - ./scripts/linux/openc3_build_ubi.sh {{.CLI_ARGS}} + + run-ubi: + desc: Run COSMOS with UBI containers + summary: | + Run OpenC3 COSMOS UBI containers in detached mode. + + Uses UBI-based container images with UBI-specific configurations. + dotenv: + - "{{.ENV_FILE}}" + env: + OPENC3_IMAGE_SUFFIX: "-ubi" + OPENC3_REDIS_VOLUME: "/home/data" + cmds: + - task: _check-root + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} up -d" + + # ============================================================================ + # CLI Commands + # ============================================================================ + + cli: + desc: Run COSMOS CLI commands in a container + summary: | + Run OpenC3 COSMOS CLI commands inside a Docker container. + + Usage: task cli -- [options] + + Examples: + task cli -- help # Show CLI help + task cli -- validate plugin.gem # Validate a plugin + task cli -- load plugin.gem # Load a plugin + task cli -- generate plugin MyPlugin # Generate new plugin + + The current directory is mounted as /openc3/local inside the container. + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - | + {{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} run -it --rm \ + -v "$(pwd):/openc3/local:z" \ + -w /openc3/local \ + -e OPENC3_API_PASSWORD=${OPENC3_API_PASSWORD:-} \ + --no-deps \ + openc3-cosmos-cmd-tlm-api ruby /openc3/bin/openc3cli {{.CLI_ARGS}} + + cliroot: + desc: Run COSMOS CLI commands as root user + summary: | + Run OpenC3 COSMOS CLI commands inside a Docker container as root. + + Same as 'cli' but runs as root user for elevated privileges. + + Usage: task cliroot -- [options] + silent: true + dotenv: + - "{{.ENV_FILE}}" + cmds: + - | + {{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} run -it --rm \ + --user=root \ + -v "$(pwd):/openc3/local:z" \ + -w /openc3/local \ + -e OPENC3_API_PASSWORD=${OPENC3_API_PASSWORD:-} \ + --no-deps \ + openc3-cosmos-cmd-tlm-api ruby /openc3/bin/openc3cli {{.CLI_ARGS}} + + cli-help: + desc: Show COSMOS CLI help + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: help + + "generate:plugin": + desc: Generate a new plugin + summary: | + Generate a new OpenC3 COSMOS plugin. + + Usage: task generate:plugin -- [--ruby|--python] + + Example: task generate:plugin -- LRO --ruby + + This creates a plugin directory structure with: + - plugin.txt (plugin configuration) + - targets// (target directory) + - Rakefile (for building the gem) + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: generate plugin {{.CLI_ARGS}} + + "generate:target": + desc: Generate a new target + summary: | + Generate a new OpenC3 COSMOS target within an existing plugin. + + Usage: task generate:target -- + + Example: task generate:target -- MY_TARGET + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: generate target {{.CLI_ARGS}} + + cli-validate: + desc: Validate a plugin + summary: | + Validate an OpenC3 COSMOS plugin gem file. + + Usage: task cli-validate -- + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: validate {{.CLI_ARGS}} + + cli-load: + desc: Load a plugin + summary: | + Load an OpenC3 COSMOS plugin. + + Usage: task cli-load -- + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: load {{.CLI_ARGS}} + + xtce-import: + desc: Convert XTCE file to COSMOS configuration + summary: | + Convert an XTCE (XML Telemetry and Command Exchange) file into + COSMOS configuration files. + + Usage: task xtce-import -- --output + + Example: task xtce-import -- satellite.xtce --output ./my_target + + The converted configuration files will be placed in a target folder + within the specified output directory. + + Note: XTCE files can also be used directly by placing them in + a target's cmd_tlm folder without conversion. + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: xtce_converter --import {{.CLI_ARGS}} + + xtce-export: + desc: Convert COSMOS plugin to XTCE format + summary: | + Convert an OpenC3 COSMOS plugin to XTCE format. + + Usage: task xtce-export -- --output + + Example: task xtce-export -- openc3-cosmos-demo.gem --output ./xtce_output + + One .xtce file will be created per target in the plugin. + silent: true + cmds: + - task: cli + vars: + CLI_ARGS: xtce_converter --plugin {{.CLI_ARGS}} + + # ============================================================================ + # Test Commands + # ============================================================================ + + test-rspec: + desc: Run RSpec tests against Ruby code + summary: | + Run RSpec tests in a Docker container. + + This builds the containers first, then runs the RSpec test suite. + dotenv: + - "{{.ENV_FILE}}" + cmds: + - task: setup + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build" + - ./scripts/linux/openc3_test.sh rspec {{.CLI_ARGS}} + + test-playwright: + desc: Run Playwright end-to-end tests + summary: | + Run Playwright end-to-end tests. + + Subcommands available: + task test-playwright-install Install playwright and dependencies + task test-playwright-build Build the plugin for tests + task test-playwright-run Run tests using Chrome + + Or run directly with arguments. + dotenv: + - "{{.ENV_FILE}}" + cmds: + - task: setup + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build" + - ./scripts/linux/openc3_test.sh playwright {{.CLI_ARGS}} + + test-playwright-install: + desc: Install Playwright and dependencies + dir: playwright + cmds: + - ./playwright.sh install-playwright + + test-playwright-build: + desc: Build the plugin for Playwright tests + dir: playwright + cmds: + - ./playwright.sh build-plugin + + test-playwright-run: + desc: Run Playwright tests + dir: playwright + cmds: + - pnpm test {{.CLI_ARGS}} + + test-hash: + desc: Run comprehensive tests with coverage + dotenv: + - "{{.ENV_FILE}}" + cmds: + - task: setup + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} -f {{.COMPOSE_BUILD_FILE}} build" + - ./scripts/linux/openc3_test.sh hash {{.CLI_ARGS}} + + # ============================================================================ + # Utility Commands + # ============================================================================ + + util-encode: + desc: Encode a string to base64 + summary: | + Encode a string to base64. + + Usage: task util-encode -- + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh encode {{.CLI_ARGS}} + + util-hash: + desc: Hash a string using SHA-256 + summary: | + Hash a string using SHA-256. + + Usage: task util-hash -- + cmds: + - ./scripts/linux/openc3_util.sh hash {{.CLI_ARGS}} + + util-save: + desc: Save Docker images to tar files + summary: | + Save OpenC3 COSMOS Docker images to tar files. + + Usage: task util-save -- [suffix] + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh save {{.CLI_ARGS}} + + util-load: + desc: Load Docker images from tar files + summary: | + Load OpenC3 COSMOS Docker images from tar files. + + Usage: task util-load -- [tag] [suffix] + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh load {{.CLI_ARGS}} + + util-tag: + desc: Tag images from one repo to another + summary: | + Tag OpenC3 COSMOS Docker images from one repository to another. + + Usage: task util-tag -- [ns2] [tag2] [suffix] + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh tag {{.CLI_ARGS}} + + util-push: + desc: Push images to Docker repository + summary: | + Push OpenC3 COSMOS Docker images to a repository. + + Usage: task util-push -- [suffix] + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh push {{.CLI_ARGS}} + + util-clean: + desc: Remove node_modules, coverage, etc. + dotenv: + - "{{.ENV_FILE}}" + cmds: + - ./scripts/linux/openc3_util.sh clean + + # ============================================================================ + # Cleanup Commands + # ============================================================================ + + cleanup: + desc: Remove all Docker volumes and data (DESTRUCTIVE!) + summary: | + Remove all OpenC3 COSMOS Docker volumes and data. + + WARNING: This is a destructive operation that removes ALL COSMOS data! + + Use 'task cleanup-force' to skip confirmation. + Use 'task cleanup-local' to also remove local plugin files. + dotenv: + - "{{.ENV_FILE}}" + prompt: Are you sure you want to remove ALL Docker volumes and COSMOS data? + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} down -t 30 -v" + + cleanup-force: + desc: Remove all Docker volumes without confirmation + dotenv: + - "{{.ENV_FILE}}" + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} down -t 30 -v" + + cleanup-local: + desc: Remove Docker volumes and local plugin files + summary: | + Remove all OpenC3 COSMOS Docker volumes and local plugin files. + + This removes: + - All Docker volumes + - Local plugin files in plugins/DEFAULT/ + dotenv: + - "{{.ENV_FILE}}" + prompt: Are you sure you want to remove ALL Docker volumes, COSMOS data, and local plugins? + cmds: + - "{{.CONTAINER_BIN}} compose -f {{.COMPOSE_FILE}} down -t 30 -v" + - | + cd plugins/DEFAULT && ls | grep -xv "README.md" | xargs rm -rf || true + + # ============================================================================ + # Ruby Development Tasks + # ============================================================================ + + ruby-install: + desc: Install Ruby dependencies + dir: openc3 + cmds: + - bundle install + + ruby-build: + desc: Compile Ruby C extensions + dir: openc3 + cmds: + - bundle exec rake build + + ruby-test: + desc: Run Ruby RSpec tests + summary: | + Run Ruby RSpec tests locally (not in Docker). + + Usage: + task ruby-test # Run all tests + task ruby-test -- spec/path/to/spec.rb # Run specific file + task ruby-test -- spec/path/to/spec.rb:42 # Run specific line + dir: openc3 + cmds: + - bundle exec rspec {{.CLI_ARGS}} + + # ============================================================================ + # Python Development Tasks + # ============================================================================ + + python-install: + desc: Install Python dependencies + dir: openc3/python + cmds: + - poetry install + + python-lint: + desc: Run Python linter (ruff) + dir: openc3/python + cmds: + - poetry run ruff check openc3 {{.CLI_ARGS}} + + python-test: + desc: Run Python tests + summary: | + Run Python pytest tests locally. + + Usage: + task python-test # Run all tests + task python-test -- test/path/to/test.py # Run specific file + dir: openc3/python + cmds: + - poetry run pytest {{.CLI_ARGS}} + + python-coverage: + desc: Run Python tests with coverage + dir: openc3/python + cmds: + - poetry run coverage run -m pytest + - poetry run coverage report + + # ============================================================================ + # Frontend Development Tasks + # ============================================================================ + + frontend-install: + desc: Install frontend dependencies + dir: openc3-cosmos-init/plugins + cmds: + - pnpm install + + frontend-build: + desc: Build frontend shared packages + dir: openc3-cosmos-init/plugins + cmds: + - pnpm build:common + + frontend-lint: + desc: Run ESLint on frontend code + summary: | + Run ESLint on frontend code. + + Run from the specific tool directory for best results. + dir: openc3-cosmos-init/plugins + cmds: + - pnpm lint {{.CLI_ARGS}} + + # ============================================================================ + # API Development Tasks + # ============================================================================ + + api-cmd-tlm-test: + desc: Run cmd-tlm-api RSpec tests + dir: openc3-cosmos-cmd-tlm-api + cmds: + - bundle exec rspec {{.CLI_ARGS}} + + api-script-runner-test: + desc: Run script-runner-api RSpec tests + dir: openc3-cosmos-script-runner-api + cmds: + - bundle exec rspec {{.CLI_ARGS}} + + # ============================================================================ + # Git Hooks + # ============================================================================ + + hooks-install: + desc: Install git hooks + cmds: + - ./hooks/install.sh + + # ============================================================================ + # Internal Tasks + # ============================================================================ + + _check-root: + internal: true + silent: true + cmds: + - | + if [ "$(id -u)" -eq 0 ]; then + echo "WARNING: COSMOS should not be run as the root user, as permissions for Local Mode will be affected." + echo "Do not use sudo when running COSMOS. See more: https://docs.openc3.com/docs/guides/local-mode" + fi diff --git a/architecture.md b/architecture.md new file mode 100644 index 0000000000..3dd75306a8 --- /dev/null +++ b/architecture.md @@ -0,0 +1,215 @@ +# OpenC3 COSMOS Architecture + +This document describes the architecture of OpenC3 COSMOS, a command and control system for embedded systems. + +## System Overview + +OpenC3 COSMOS is deployed as a set of Docker containers that work together to provide a web-based interface for telemetry display, command sending, script execution, logging, and more. + +## Architecture Diagram + +```mermaid +flowchart TB + subgraph external["External Access"] + User["👤 User Browser"] + end + + subgraph ports["Exposed Ports"] + P2900["Port 2900
(HTTP)"] + P2943["Port 2943
(HTTPS)"] + P9000["Port 9000
(TSDB Console)"] + end + + subgraph services["Docker Services"] + subgraph proxy["Reverse Proxy"] + traefik["openc3-traefik
━━━━━━━━━━━
Traefik Proxy"] + end + + subgraph apis["API Layer"] + cmdtlm["openc3-cosmos-cmd-tlm-api
━━━━━━━━━━━
Rails 7.2 REST API
Command/Telemetry"] + scriptrunner["openc3-cosmos-script-runner-api
━━━━━━━━━━━
Rails 7.2 API
Script Execution"] + end + + subgraph core["Core Services"] + operator["openc3-operator
━━━━━━━━━━━
Ruby Operator
Manages Interfaces"] + init["openc3-cosmos-init
━━━━━━━━━━━
Initialization
(runs once)"] + end + + subgraph data["Data Layer"] + redis["openc3-redis
━━━━━━━━━━━
Redis
Persistent State"] + redis_eph["openc3-redis-ephemeral
━━━━━━━━━━━
Redis
Ephemeral State"] + buckets["openc3-buckets
━━━━━━━━━━━
S3-Compatible
Object Storage"] + tsdb["openc3-tsdb
━━━━━━━━━━━
QuestDB
Time Series DB"] + end + end + + subgraph volumes["Docker Volumes"] + v_redis["openc3-redis-v"] + v_redis_eph["openc3-redis-ephemeral-v"] + v_object["openc3-object-v"] + v_gems["openc3-gems-v"] + v_tsdb["openc3-tsdb-v"] + end + + %% External connections + User --> P2900 + User --> P2943 + P2900 --> traefik + P2943 --> traefik + P9000 --> tsdb + + %% Traefik routes to APIs + traefik -.-> cmdtlm + traefik -.-> scriptrunner + + %% Service dependencies + traefik --> redis + traefik --> redis_eph + traefik --> buckets + + cmdtlm --> redis + cmdtlm --> redis_eph + cmdtlm --> buckets + + scriptrunner --> redis + scriptrunner --> redis_eph + scriptrunner --> buckets + + operator --> redis + operator --> redis_eph + operator --> buckets + + init --> traefik + init --> redis + init --> redis_eph + init --> buckets + + %% Volume connections + redis -.-> v_redis + redis_eph -.-> v_redis_eph + buckets -.-> v_object + tsdb -.-> v_tsdb + cmdtlm -.-> v_gems + scriptrunner -.-> v_gems + operator -.-> v_gems + init -.-> v_gems + + %% Styling + classDef proxy fill:#e1f5fe,stroke:#01579b + classDef api fill:#fff3e0,stroke:#e65100 + classDef core fill:#f3e5f5,stroke:#7b1fa2 + classDef data fill:#e8f5e9,stroke:#2e7d32 + classDef volume fill:#fafafa,stroke:#616161 + classDef port fill:#ffebee,stroke:#c62828 + + class traefik proxy + class cmdtlm,scriptrunner api + class operator,init core + class redis,redis_eph,buckets,tsdb data + class v_redis,v_redis_eph,v_object,v_gems,v_tsdb volume + class P2900,P2943,P9000 port +``` + +## Service Descriptions + +### Proxy Layer + +| Service | Description | +|---------|-------------| +| **openc3-traefik** | Traefik reverse proxy that handles all incoming HTTP/HTTPS traffic on ports 2900 and 2943. Routes requests to appropriate backend services. | + +### API Layer + +| Service | Description | +|---------|-------------| +| **openc3-cosmos-cmd-tlm-api** | Rails 7.2 REST API for command and telemetry operations. Handles WebSocket connections via AnyCable for real-time updates. | +| **openc3-cosmos-script-runner-api** | Rails 7.2 API for script execution. Manages running Ruby and Python scripts for automation. | + +### Core Services + +| Service | Description | +|---------|-------------| +| **openc3-operator** | Ruby-based operator that manages interfaces and microservices. Spawns and monitors target interfaces, routers, and other dynamic components. | +| **openc3-cosmos-init** | Initialization container that runs once on startup. Loads plugins, tools, and initial configuration into the system. | + +### Data Layer + +| Service | Description | +|---------|-------------| +| **openc3-redis** | Redis instance for persistent state storage. Stores configuration, command/telemetry definitions, and system state. | +| **openc3-redis-ephemeral** | Redis instance for ephemeral data. Handles pub/sub messaging and temporary state that doesn't need persistence. | +| **openc3-buckets** | S3-compatible object storage (Versitygw). Stores logs, configuration files, and plugin artifacts. | +| **openc3-tsdb** | QuestDB time series database. Stores historical telemetry data for analysis and graphing. | + +## Docker Volumes + +| Volume | Purpose | +|--------|---------| +| **openc3-redis-v** | Persistent Redis data | +| **openc3-redis-ephemeral-v** | Ephemeral Redis data | +| **openc3-object-v** | S3 bucket object storage | +| **openc3-gems-v** | Shared Ruby gems and Python packages | +| **openc3-tsdb-v** | QuestDB time series data | + +## Network Ports + +| Port | Protocol | Service | Description | +|------|----------|---------|-------------| +| 2900 | HTTP | openc3-traefik | Main web interface | +| 2943 | HTTPS | openc3-traefik | Secure web interface | +| 9000 | HTTP | openc3-tsdb | QuestDB console and ingest | + +## Service Dependencies + +``` +openc3-cosmos-init +├── openc3-traefik +│ ├── openc3-redis +│ ├── openc3-redis-ephemeral +│ └── openc3-buckets +├── openc3-redis +├── openc3-redis-ephemeral +└── openc3-buckets + +openc3-cosmos-cmd-tlm-api +├── openc3-redis +├── openc3-redis-ephemeral +└── openc3-buckets + +openc3-cosmos-script-runner-api +├── openc3-redis +├── openc3-redis-ephemeral +└── openc3-buckets + +openc3-operator +├── openc3-redis +├── openc3-redis-ephemeral +└── openc3-buckets +``` + +## Communication Patterns + +1. **HTTP/REST**: External clients communicate with the system via Traefik on port 2900/2943 +2. **WebSocket**: Real-time updates use AnyCable backed by Redis pub/sub +3. **Redis Pub/Sub**: Internal service-to-service messaging for events and state changes +4. **S3 Protocol**: Log and configuration file storage via the buckets service + +## Startup Sequence + +1. **Data layer starts first**: Redis, Redis Ephemeral, Buckets, and TSDB containers start +2. **Proxy starts**: Traefik starts and waits for data layer +3. **Init runs**: openc3-cosmos-init loads plugins and initial configuration +4. **APIs start**: cmd-tlm-api and script-runner-api become available +5. **Operator starts**: Spawns configured interfaces and microservices + +## Technology Stack + +- **Ruby 3.4** - Backend APIs and core library +- **Rails 7.2** - REST APIs with AnyCable for WebSockets +- **Python 3.10-3.12** - Alternative scripting language +- **Vue.js 3 + Vuetify 3** - Frontend UI framework +- **Redis** - Caching, pub/sub, state management +- **QuestDB** - Time series database +- **Versitygw** - S3-compatible object storage +- **Traefik** - Reverse proxy and load balancer +- **Docker Compose** - Container orchestration