diff --git a/README.md b/README.md index 2145ae9d..dc89d4de 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@
-![dstack](./dstack-logo.svg) +![dstack-cloud](./dstack-logo.svg) -### The open framework for confidential AI. +### Deploy confidential workloads on GCP and AWS. [![GitHub Stars](https://img.shields.io/github/stars/dstack-tee/dstack?style=flat-square&logo=github)](https://github.com/Phala-Network/dstack-cloud/stargazers) [![License](https://img.shields.io/github/license/dstack-tee/dstack?style=flat-square)](https://github.com/Phala-Network/dstack-cloud/blob/master/LICENSE) @@ -16,40 +16,33 @@ --- -## What is dstack? +## What is dstack-cloud? -dstack is the open framework for confidential AI - deploy AI applications with cryptographic privacy guarantees. +dstack-cloud extends [dstack](https://github.com/Dstack-TEE/dstack) to deploy containers on **GCP Confidential VMs** and **AWS Nitro Enclaves**. It provisions the VM, manages attestation, and handles networking. You get confidential computing on cloud infrastructure without running your own TDX hardware. -AI providers ask users to trust them with sensitive data. But trust doesn't scale, and trust can't be verified. With dstack, your containers run inside confidential VMs (Intel TDX) with native support for NVIDIA Confidential Computing (H100, Blackwell). Users can cryptographically verify exactly what's running: private AI with your existing Docker workflow. +Your containers run with full security infrastructure out of the box: key management, remote attestation, hardened OS, and encrypted storage. Users can cryptographically verify exactly what's running. -### What is dstack-cloud? +## Supported Platforms -`dstack-cloud` is a commercial distribution of dstack. It primarily extends the open framework with additional support for cloud platforms such as Google Cloud (GCP) and AWS Nitro Enclaves. +| Platform | Status | Attestation | +|----------|--------|-------------| +| **[Phala Cloud](https://cloud.phala.network)** | Available | TDX | +| **GCP Confidential VMs** | Available | TDX + TPM | +| **AWS Nitro Enclaves** | Available | NSM | +| **Bare metal TDX** | Available | TDX | -### Features +## Quick Start -**Zero friction onboarding** -- **Docker Compose native**: Bring your docker-compose.yaml as-is. No SDK, no code changes. -- **Encrypted by default**: Network traffic and disk storage encrypted out of the box. - -**Hardware-rooted security** -- **Private by hardware**: Data encrypted in memory, inaccessible even to the host. -- **Reproducible OS**: Deterministic builds mean anyone can verify the OS image hash. -- **Workload identity**: Every app gets an attested identity users can verify cryptographically. -- **Confidential GPUs**: Native support for NVIDIA Confidential Computing (H100, Blackwell). - -**Trustless operations** -- **Isolated keys**: Per-app keys derived in TEE. Survives hardware failure. Never exposed to operators. -- **Code governance**: Updates follow predefined rules (e.g., multi-party approval). Operators can't swap code or access secrets. +**1. Create a project:** -## Getting Started - -**Try it now:** Chat with LLMs running in TEE at [chat.redpill.ai](https://chat.redpill.ai). Click the shield icon to verify attestations from Intel TDX and NVIDIA GPUs. +```bash +dstack-cloud new my-app +cd my-app +``` -**Deploy your own:** +**2. Edit your docker-compose.yaml:** ```yaml -# docker-compose.yaml services: vllm: image: vllm/vllm-openai:latest @@ -59,54 +52,105 @@ services: - "8000:8000" ``` -Deploy to any TDX host with the [`dstack-nvidia-0.6.x` base image](https://github.com/Phala-Network/meta-dstack-cloud/releases), or use [Phala Cloud](https://cloud.phala.network) for managed infrastructure. +**3. Deploy:** + +```bash +dstack-cloud deploy +``` + +**4. Check status:** + +```bash +dstack-cloud status +dstack-cloud logs --follow +``` + +For the full walkthrough, see the [Quickstart Guide](./docs/quickstart.md). + +## Features -Want to deploy a self hosted dstack? Check our [full deployment guide →](./docs/deployment.md) +**Zero friction onboarding** +- **Docker Compose native**: Bring your docker-compose.yaml as-is. No SDK, no code changes. +- **Encrypted by default**: Network traffic and disk storage encrypted out of the box. + +**Hardware-rooted security** +- **Private by hardware**: Data encrypted in memory, inaccessible even to the host. +- **Reproducible OS**: Deterministic builds mean anyone can verify the OS image hash. +- **Workload identity**: Every app gets an attested identity users can verify cryptographically. +- **Confidential GPUs**: Native support for NVIDIA Confidential Computing (H100, Blackwell). + +**Trustless operations** +- **Isolated keys**: Per-app keys derived in TEE. Survives hardware failure. Never exposed to operators. +- **Code governance**: Updates follow predefined rules (e.g., multi-party approval). Operators can't swap code or access secrets. ## Architecture ![Architecture](./docs/assets/arch.png) -Your container runs inside a Confidential VM (Intel TDX) with optional GPU isolation via NVIDIA Confidential Computing. The CPU TEE protects application logic; the GPU TEE protects model weights and inference data. +Your container runs inside a Confidential VM (Intel TDX on GCP, Nitro Enclave on AWS). GPU isolation is optional via NVIDIA Confidential Computing. The CPU TEE protects application logic. The GPU TEE protects model weights and inference data. **Core components:** -- **Guest Agent**: Runs inside each CVM. Generates TDX attestation quotes so users can verify exactly what's running. Provisions per-app cryptographic keys from KMS. Encrypts local storage. Apps interact via `/var/run/dstack.sock`. +- **Guest Agent**: Runs inside each CVM. Generates attestation quotes so users can verify exactly what's running. Provisions per-app cryptographic keys from KMS. Encrypts local storage. Apps interact via `/var/run/dstack.sock`. -- **KMS**: Runs in its own TEE. Verifies TDX quotes before releasing keys. Enforces authorization policies defined in on-chain smart contracts — operators cannot bypass these checks. Derives deterministic keys bound to each app's attested identity. +- **KMS**: Runs in its own TEE. Verifies attestation quotes before releasing keys. Enforces authorization policies that operators cannot bypass. Derives deterministic keys bound to each app's attested identity. -- **Gateway**: Terminates TLS at the edge and provisions ACME certificates automatically. Routes traffic to CVMs. All internal communication uses RA-TLS for mutual attestation. +- **Gateway**: Terminates TLS at the edge. Provisions ACME certificates automatically. Routes traffic to CVMs. Internal communication uses RA-TLS for mutual attestation. -- **VMM**: Runs on bare-metal TDX hosts. Parses docker-compose files directly — no app changes needed. Boots CVMs from a reproducible OS image. Allocates CPU, memory, and confidential GPU resources. +- **VMM**: Parses docker-compose files directly — no app changes needed. Boots CVMs from a reproducible OS image. Allocates CPU, memory, and confidential GPU resources. [Full security model →](./docs/security/security-model.md) +## CLI Reference + +``` +dstack-cloud new # Create a new project +dstack-cloud config-edit # Edit global configuration +dstack-cloud deploy # Deploy to cloud +dstack-cloud status # Check deployment status +dstack-cloud logs [--follow] # View console logs +dstack-cloud stop # Stop the VM +dstack-cloud start # Start a stopped VM +dstack-cloud remove # Remove the VM and cleanup +dstack-cloud list # List all deployments +dstack-cloud fw allow # Allow traffic on a port +dstack-cloud fw deny # Block traffic on a port +dstack-cloud fw list # List firewall rules +``` + ## SDKs -Apps communicate with the guest agent via HTTP over `/var/run/dstack.sock`. Use the [HTTP API](./sdk/curl/api.md) directly with curl, or use a language SDK: +Apps communicate with the guest agent via HTTP over `/var/run/dstack.sock`. Use the [HTTP API](https://github.com/Dstack-TEE/dstack/blob/master/sdk/curl/api.md) directly with curl, or use a language SDK: | Language | Install | Docs | |----------|---------|------| -| Python | `pip install dstack-sdk` | [README](./sdk/python/README.md) | -| TypeScript | `npm install @phala/dstack-sdk` | [README](./sdk/js/README.md) | -| Rust | `cargo add dstack-sdk` | [README](./sdk/rust/README.md) | -| Go | `go get github.com/Dstack-TEE/dstack/sdk/go` | [README](./sdk/go/README.md) | +| Python | `pip install dstack-sdk` | [README](https://github.com/Dstack-TEE/dstack/blob/master/sdk/python/README.md) | +| TypeScript | `npm install @phala/dstack-sdk` | [README](https://github.com/Dstack-TEE/dstack/blob/master/sdk/js/README.md) | +| Rust | `cargo add dstack-sdk` | [README](https://github.com/Dstack-TEE/dstack/blob/master/sdk/rust/README.md) | +| Go | `go get github.com/Dstack-TEE/dstack/sdk/go` | [README](https://github.com/Dstack-TEE/dstack/blob/master/sdk/go/README.md) | ## Documentation -**For Developers** -- [Confidential AI](./docs/confidential-ai.md) - Inference, agents, and training with hardware privacy +**Getting Started** +- [Quickstart](./docs/quickstart.md) - Deploy your first app on GCP or AWS - [Usage Guide](./docs/usage.md) - Deploying and managing apps - [Verification](./docs/verification.md) - How to verify TEE attestation -**For Operators** +**Cloud Platforms** +- [GCP Attestation](./docs/attestation-gcp.md) - TDX + TPM attestation on GCP +- [AWS Nitro Attestation](./docs/attestation-nitro-enclave.md) - NSM attestation on AWS + +**For Developers** +- [Confidential AI](./docs/confidential-ai.md) - Inference, agents, and training with hardware privacy +- [App Compose Format](./docs/normalized-app-compose.md) - Compose file specification + +**Self-Hosted / Bare Metal** - [Deployment](./docs/deployment.md) - Self-hosting on TDX hardware -- [On-Chain Governance](./docs/onchain-governance.md) - Smart contract authorization +- [VMM CLI Guide](./docs/vmm-cli-user-guide.md) - VMM command-line reference - [Gateway](./docs/dstack-gateway.md) - Gateway configuration +- [On-Chain Governance](./docs/onchain-governance.md) - Policy-based authorization **Reference** -- [App Compose Format](./docs/normalized-app-compose.md) - Compose file specification -- [VMM CLI Guide](./docs/vmm-cli-user-guide.md) - Command-line reference - [Design Decisions](./docs/design-and-hardening-decisions.md) - Architecture rationale - [FAQ](./docs/faq.md) - Frequently asked questions @@ -123,51 +167,56 @@ Apps communicate with the guest agent via HTTP over `/var/run/dstack.sock`. Use
Why not use AWS Nitro / Azure Confidential VMs / GCP directly? -You can — but you'll build everything yourself: attestation verification, key management, Docker orchestration, certificate provisioning, and governance. dstack provides all of this out of the box. +You can — but you'll build everything yourself: attestation verification, key management, Docker orchestration, certificate provisioning, and governance. dstack-cloud provides all of this out of the box. | Approach | Docker native | GPU TEE | Key management | Attestation tooling | Open source | |----------|:-------------:|:-------:|:--------------:|:-------------------:|:-----------:| -| **dstack** | ✓ | ✓ | ✓ | ✓ | ✓ | +| **dstack-cloud** | ✓ | ✓ | ✓ | ✓ | ✓ | | AWS Nitro Enclaves | - | - | Manual | Manual | - | | Azure Confidential VMs | - | Preview | Manual | Manual | - | | GCP Confidential Computing | - | - | Manual | Manual | - | -Cloud providers give you the hardware primitive. dstack gives you the full stack: reproducible OS images, automatic attestation, per-app key derivation, TLS certificates, and smart contract governance. No vendor lock-in. +Cloud providers give you the hardware primitive. dstack-cloud gives you the full stack: reproducible OS images, automatic attestation, per-app key derivation, and TLS certificates. No vendor lock-in.
How is this different from SGX/Gramine? -SGX requires porting applications to enclaves. dstack uses full-VM isolation (Intel TDX) — bring your Docker containers as-is. Plus GPU TEE support that SGX doesn't offer. +SGX requires porting applications to enclaves. dstack-cloud uses full-VM isolation (Intel TDX, AWS Nitro) — bring your Docker containers as-is. Plus GPU TEE support that SGX doesn't offer.
What's the performance overhead? -Minimal. Intel TDX adds ~2-5% overhead for CPU workloads. NVIDIA Confidential Computing has negligible impact on GPU inference. The main cost is memory encryption, which is hardware-accelerated on supported CPUs. +Minimal. Intel TDX adds ~2-5% overhead for CPU workloads. NVIDIA Confidential Computing has negligible impact on GPU inference. Memory encryption is the main cost, but it's hardware-accelerated on supported CPUs.
Is this production-ready? -Yes. dstack powers production AI infrastructure at [OpenRouter](https://openrouter.ai/provider/phala) and [NEAR AI](https://x.com/ilblackdragon/status/1962920246148268235). The framework has been [audited by zkSecurity](./docs/security/dstack-audit.pdf) and is a Linux Foundation Confidential Computing Consortium project. +Yes. dstack powers production AI at [OpenRouter](https://openrouter.ai/provider/phala) and [NEAR AI](https://x.com/ilblackdragon/status/1962920246148268235). It's been [audited by zkSecurity](./docs/security/dstack-audit.pdf). It's a Linux Foundation Confidential Computing Consortium project.
Can I run this on my own hardware? -Yes. dstack runs on any Intel TDX-capable server. See the [deployment guide](./docs/deployment.md) for self-hosting instructions. You can also use [Phala Cloud](https://cloud.phala.network) for managed infrastructure. +Yes. dstack-cloud runs on any Intel TDX-capable server. See the [deployment guide](./docs/deployment.md) for self-hosting instructions. You can also use [Phala Cloud](https://cloud.phala.network) for managed infrastructure.
What TEE hardware is supported? -Currently: Intel TDX (4th/5th Gen Xeon) and NVIDIA Confidential Computing (H100, Blackwell). AMD SEV-SNP support is planned. +- **GCP**: Intel TDX (Confidential VMs) +- **AWS**: Nitro Enclaves (NSM attestation) +- **Bare metal**: Intel TDX (4th/5th Gen Xeon) +- **GPUs**: NVIDIA Confidential Computing (H100, Blackwell) + +AMD SEV-SNP support is planned.
@@ -189,6 +238,8 @@ dstack is a Linux Foundation [Confidential Computing Consortium](https://confide [Telegram](https://t.me/+UO4bS4jflr45YmUx) · [GitHub Discussions](https://github.com/Dstack-TEE/dstack/discussions) · [Examples](https://github.com/Dstack-TEE/dstack-examples) +For enterprise support and licensing, [book a call](https://cal.com/team/phala/founders) or email us at support@phala.network. + [![Repobeats](https://repobeats.axiom.co/api/embed/0a001cc3c1f387fae08172a9e116b0ec367b8971.svg)](https://github.com/Dstack-TEE/dstack/pulse) ## Cite @@ -212,4 +263,4 @@ Logo and branding assets: [dstack-logo-kit](./docs/assets/dstack-logo-kit/) This repository is licensed under the Business Source License 1.1 (BUSL-1.1). Per the terms in [LICENSE](./LICENSE), the Licensed Work is `dstack-cloud`. -By default, BUSL-1.1 permits copying, modification, redistribution, and **non-production** use. The license does not grant production use unless an Additional Use Grant is provided (this project specifies: None). If your intended use does not comply with BUSL-1.1, you must obtain a commercial license from Phala Network (or authorized resellers). +BUSL-1.1 permits copying, modification, redistribution, and **non-production** use. Production use requires a commercial license. [Book a call](https://cal.com/team/phala/founders) or email support@phala.network. diff --git a/docs/attestation-nitro-enclave.md b/docs/attestation-nitro-enclave.md index 52881c64..1173cb75 100644 --- a/docs/attestation-nitro-enclave.md +++ b/docs/attestation-nitro-enclave.md @@ -1,5 +1,7 @@ # Dstack Nitro Enclave Attestation Flow (NSM) +> **AWS Nitro Support:** Attestation verification is fully implemented. For AWS deployment options, [book a call](https://calendly.com/aspect-ux/30min) with our team. + This document describes how dstack produces and verifies attestation on AWS Nitro Enclaves using the NSM attestation document. It follows the implementation in `dstack-attest` and `nsm-qvl`. diff --git a/docs/auth-simple-operations.md b/docs/auth-simple-operations.md index 8b253b9c..88bfd7d6 100644 --- a/docs/auth-simple-operations.md +++ b/docs/auth-simple-operations.md @@ -1,5 +1,7 @@ # auth-simple Operations Guide +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). + This guide covers day-to-day operations for managing apps and devices with auth-simple. For initial deployment setup, see [Deployment Guide](./deployment.md). diff --git a/docs/deployment.md b/docs/deployment.md index 9c5872d3..8c9c5916 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -1,5 +1,7 @@ # Deploying dstack +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). + This guide covers deploying dstack on bare metal TDX hosts. ## Overview diff --git a/docs/dstack-gateway.md b/docs/dstack-gateway.md index be1e2cf4..b7a36201 100644 --- a/docs/dstack-gateway.md +++ b/docs/dstack-gateway.md @@ -1,5 +1,7 @@ # Setup dstack-gateway for Production +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). + To set up dstack-gateway for production, you need a wildcard domain and SSL certificate. ## Step 1: Setup wildcard domain diff --git a/docs/onchain-governance.md b/docs/onchain-governance.md index 67497f8b..b4700c13 100644 --- a/docs/onchain-governance.md +++ b/docs/onchain-governance.md @@ -1,5 +1,7 @@ # On-Chain Governance +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). + This guide covers setting up on-chain governance for dstack using smart contracts on Ethereum. ## Overview diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 00000000..c94dc357 --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,209 @@ +# Quickstart + +Deploy your first confidential workload on GCP in under 10 minutes. + +> **Interested in AWS Nitro Enclaves?** We support AWS Nitro attestation verification and are expanding deployment tooling. [Book a call](https://calendly.com/aspect-ux/30min) to learn more about AWS deployment options. + +## Prerequisites + +- GCP account with Confidential VM quota (Intel TDX) +- `gcloud` CLI installed and authenticated + +## Install the CLI + +Download the `dstack-cloud` CLI: + +```bash +# Clone the repository (temporary until packaged release) +git clone https://github.com/Phala-Network/meta-dstack-cloud.git +export PATH="$PATH:$(pwd)/meta-dstack-cloud/scripts/bin" +``` + +Verify the installation: + +```bash +dstack-cloud --help +``` + +## Configure + +Set up your cloud credentials: + +```bash +dstack-cloud config-edit +``` + +This opens an editor with the global configuration file. For GCP, configure: + +```toml +[gcp] +project = "your-gcp-project-id" +zone = "us-central1-a" +machine_type = "n2d-standard-4" +``` + +## Create a Project + +Create a new dstack-cloud project: + +```bash +dstack-cloud new my-app +cd my-app +``` + +This creates a project directory with: + +``` +my-app/ +├── app.json # Application configuration +├── docker-compose.yaml # Your container definition +├── .env # Environment variables +└── prelaunch.sh # Pre-launch script (optional) +``` + +## Define Your Workload + +Edit `docker-compose.yaml` with your application: + +```yaml +services: + web: + image: nginx:latest + ports: + - "8080:80" +``` + +For AI workloads with GPU: + +```yaml +services: + vllm: + image: vllm/vllm-openai:latest + runtime: nvidia + command: --model Qwen/Qwen2.5-7B-Instruct + ports: + - "8000:8000" +``` + +## Add Secrets (Optional) + +Add sensitive environment variables to `.env`: + +```bash +API_KEY=your-secret-key +DATABASE_URL=postgres://... +``` + +These are encrypted before leaving your machine and only decrypted inside the TEE. + +## Deploy + +Deploy to your cloud provider: + +```bash +dstack-cloud deploy +``` + +The CLI will: +1. Build and push your container configuration +2. Create a Confidential VM +3. Boot the dstack guest OS +4. Start your containers + +## Check Status + +Monitor your deployment: + +```bash +# Check deployment status +dstack-cloud status + +# View console logs +dstack-cloud logs + +# Follow logs in real-time +dstack-cloud logs --follow +``` + +## Configure Firewall + +Allow traffic to your application: + +```bash +# Allow HTTPS traffic +dstack-cloud fw allow 443 + +# Allow your app port +dstack-cloud fw allow 8080 + +# List firewall rules +dstack-cloud fw list +``` + +## Access Your App + +Once deployed, access your application via the assigned endpoint. The `dstack-cloud status` command shows the public URL. + +For apps with TLS: +``` +https://. +``` + +For specific ports: +``` +https://-8080. +``` + +## Verify Attestation + +Users can verify your deployment is running in a genuine TEE: + +```bash +# Get attestation quote from your app +curl https:///attestation + +# Verify with dstack-verifier +dstack-verifier verify +``` + +See the [Verification Guide](./verification.md) for details. + +## Manage Deployments + +```bash +# List all deployments +dstack-cloud list + +# Stop a deployment +dstack-cloud stop + +# Start a stopped deployment +dstack-cloud start + +# Remove a deployment completely +dstack-cloud remove +``` + +## Next Steps + +- [Usage Guide](./usage.md) - Detailed deployment and management +- [Confidential AI](./confidential-ai.md) - Run AI workloads with hardware privacy +- [GCP Attestation](./attestation-gcp.md) - How TDX + TPM attestation works +- [AWS Nitro Attestation](./attestation-nitro-enclave.md) - How NSM attestation works +- [Security Model](./security/security-model.md) - Understand the trust boundaries + +## Troubleshooting + +**Deployment stuck at "Creating VM":** +- Check your cloud quota for Confidential VMs +- Verify your credentials with `gcloud auth list` + +**Container not starting:** +- Check logs with `dstack-cloud logs` +- Verify your docker-compose.yaml syntax +- Ensure images are accessible from the cloud region + +**Cannot access application:** +- Check firewall rules with `dstack-cloud fw list` +- Verify the port mapping in docker-compose.yaml +- Check if the container is healthy in the logs diff --git a/docs/usage.md b/docs/usage.md index 64684c6c..10a10126 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -1,8 +1,10 @@ # dstack Usage Guide -This guide covers deploying and managing applications on dstack. For infrastructure setup and self-hosting, see the [Deployment Guide](./deployment.md). +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). -You can manage VMs via the dashboard or [CLI](./vmm-cli-user-guide.md). +This guide covers deploying and managing applications on self-hosted dstack infrastructure. For initial setup, see the [Deployment Guide](./deployment.md). + +You can manage VMs via the VMM dashboard or [CLI](./vmm-cli-user-guide.md). ## Deploy an App diff --git a/docs/vmm-cli-user-guide.md b/docs/vmm-cli-user-guide.md index 5befa43a..bb93315d 100644 --- a/docs/vmm-cli-user-guide.md +++ b/docs/vmm-cli-user-guide.md @@ -1,5 +1,7 @@ # VMM CLI User Guide +> **This guide is for self-hosted deployments** on your own TDX hardware. For cloud deployments, see [Quickstart](./quickstart.md). + Welcome to the **VMM CLI**! This tool helps you manage CVMs in the dstack platform. ## Table of Contents