Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,7 @@ plan
# transient log files
*.log
*.list
*.json
*.json

# macOS
.DS_Store
46 changes: 38 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ flowchart TB
subgraph GCP["GCP Project"]
CGW["Connect Gateway<br/>(IAM-auth, no public endpoint)"]

subgraph VPC["VPC (private, egress: TCP 443 only)"]
subgraph VPC["VPC (private, egress: TCP 443 + Redis 6378)"]
GW["GKE Gateway<br/>Google-managed TLS"]
PODS["Socket Firewall pods<br/>ClusterIP :80, replicas 2"]
REDIS["Memorystore Redis<br/>verdict cache (TLS)"]
CP["Private control plane<br/>172.16.0.0/28"]
NAT["Cloud NAT"]
end
Expand All @@ -33,6 +34,7 @@ flowchart TB
CGW -.->|proxied| CP
CLIENTS -->|HTTPS| GW
GW --> PODS
PODS --> REDIS
PODS --> NAT
NAT --> UPSTREAM
SECRETS --> PODS
Expand Down Expand Up @@ -65,7 +67,7 @@ flowchart LR
CGW -.->|proxied| MASTER
```

Egress from GKE nodes is **deny-by-default** at the VPC firewall layer. Only **TCP 443** is permitted outbound (HTTPS to Socket.dev and upstream registries, and the fleet Connect agent's outbound channel to Google). Port 80 is intentionally blocked.
Egress from GKE nodes is **deny-by-default** at the VPC firewall layer. Only **TCP 443** is permitted outbound to the internet (HTTPS to Socket.dev and upstream registries, and the fleet Connect agent's outbound channel to Google), plus **TCP 6378** to the Memorystore Redis host for the shared verdict cache. Port 80 is intentionally blocked.

## Data flow

Expand All @@ -74,25 +76,49 @@ sequenceDiagram
participant Dev as Developer / CI
participant GW as GKE Gateway
participant FW as Socket Firewall Pod
participant Redis as Memorystore Redis
participant Up as Socket.dev API / Registry

Dev->>GW: GET https://sfw.example.com/npm/...
GW->>FW: HTTP :80 (public TLS terminated at gateway)
FW->>Up: Scan (Socket.dev) + proxy download (registry)
Up-->>FW: Verdict + package artifact
FW->>Redis: Lookup cached verdict
alt Fresh / stale cache hit on API failure
Redis-->>FW: Last known-good verdict
else Cache miss or revalidation
FW->>Up: Scan (Socket.dev) + proxy download (registry)
Up-->>FW: Verdict + package artifact
FW->>Redis: Store verdict (TTL 24h stale window)
end
FW-->>Dev: Scanned / allowed response
```

## Fail-open and circuit breaker

When the Socket API is degraded, the firewall's circuit breaker trips. With the previous `fail_open: false` and no shared cache, that defaulted to **blocking** packages.

Current behavior (Helm values in [`helm.tf`](terraform/helm.tf)):

| Setting | Value | Effect |
|---------|-------|--------|
| `socket.failOpen` | `true` | On API/breaker failure, allow when no better signal exists |
| `socket.failOpenUnscanned` | `false` | Unscanned/unknown packages stay blocked |
| `socket.cacheTtl` | `600` | Fresh verdict window (10 minutes) |
| `redis.enabled` / `redis.ttl` | `true` / `86400` | Shared Memorystore cache; stale window 24 hours |
| `extraConfig.resilience.circuit_breaker.enabled` | `true` | Prefer cached verdicts when the breaker opens |

Stale-while-revalidate: fresh hits return immediately; between `cacheTtl` and `redis.ttl` the firewall revalidates with Socket and falls back to the stale verdict if the API (or breaker) fails. After `redis.ttl` the key expires and the next request must fetch fresh (or fail-open if still down).

## Components

| Layer | Resource | Purpose |
|-------|----------|---------|
| **Network** | VPC + subnet + secondary ranges | Isolated network; subnet has VPC flow logs |
| **Egress** | Deny-all + allow TCP 443 + Cloud NAT | Default-deny egress; HTTPS-only outbound via NAT scoped to the subnet |
| **Egress** | Deny-all + allow TCP 443 + Redis 6378 + Cloud NAT | Default-deny egress; HTTPS to internet via NAT; TLS Redis to Memorystore |
| **Compute** | Private GKE cluster + node pool | Shielded nodes, deletion protection, Calico NetworkPolicy, Binary Authorization |
| **Encryption** | Cloud KMS key ring (3 keys) | CMEK for etcd secrets, node boot disks, and Secret Manager |
| **Access** | Fleet membership + Connect Gateway | IAM-authenticated proxy to the private control plane; no public endpoint, bastion, or IAP tunnel |
| **App** | Helm `socket-firewall` | Package firewall with path-based routing, HPA, pod anti-affinity, PDB |
| **App** | Helm `socket-firewall` | Package firewall with path-based routing, HPA, pod anti-affinity, PDB; Redis-backed verdict cache with fail-open on breaker trip |
| **Cache** | Memorystore Redis (AUTH + TLS) | Shared stale-while-revalidate cache across replicas so a tripped Socket API circuit breaker serves last known-good verdicts |
| **Exposure** | GKE Gateway (`gke-l7-global-external-managed`) | External HTTPS when `firewall_domain` is set; `LoadBalancer` Service fallback when domain is unset |
| **Secrets** | Secret Manager (CMEK) → K8s secret | `SOCKET_SECURITY_API_TOKEN` for Socket.dev |
| **TLS** | Certificate Manager + GKE Gateway + SSL policy | Google-managed cert; HTTPS terminates at the LB (TLS 1.2 minimum) |
Expand All @@ -116,12 +142,13 @@ sequenceDiagram
|---------|----------------|
| **Encryption at rest** | CMEK for GKE etcd, node disks, and Secret Manager (90-day key rotation) |
| **Encryption in transit** | GCP-managed TLS at the Gateway; SSL policy enforces RESTRICTED cipher suites and TLS 1.2+ |
| **Network egress** | VPC firewall deny-all with explicit TCP 443 allow; Kubernetes egress governed by VPC rules |
| **Network egress** | VPC firewall deny-all with explicit TCP 443 (internet) and TCP 6378 (Memorystore) allows; Kubernetes egress governed by VPC rules |
| **Network ingress** | Calico NetworkPolicies default-deny ingress in the firewall namespace |
| **Image admission** | Binary Authorization (`PROJECT_SINGLETON_POLICY_ENFORCE`) |
| **Node hardening** | Shielded VMs, dedicated node SA (no `cloud-platform` scope), Workload Identity, legacy metadata endpoints disabled |
| **Control plane** | Private endpoint only — no public API server; reachable solely via Connect Gateway (IAM-authenticated) |
| **Availability** | HPA, pod anti-affinity, PodDisruptionBudget (`minAvailable: 1`), 2-node minimum. **Zonal cluster** (`us-central1-a`) — these protect against single-node loss, not a full-zone outage, and the control plane has no HA SLA |
| **Verdict cache** | Memorystore Redis (AUTH + in-transit TLS); fresh TTL 10m, stale window 24h; `fail_open: true` so a tripped Socket API circuit breaker serves cached verdicts instead of blocking |
| **IAM least privilege** | Custom Terraform roles replace `container.admin`/`secretmanager.secretAdmin`/`cloudkms.admin` (no project-wide secret payload access); read-only plan SA distinct from write apply SA; fleet write access is bootstrap-only |
| **Audit** | VPC flow logs and firewall rule logging with full metadata |

Expand Down Expand Up @@ -163,7 +190,8 @@ The Gateway terminates the public, browser-trusted certificate and forwards to t
| [`fleet.tf`](terraform/fleet.tf) | Fleet (GKE Hub) membership enabling Connect Gateway access to the private control plane |
| [`iam.tf`](terraform/iam.tf) | GKE node SA, custom least-privilege Terraform roles (apply + plan SA), IAM bindings |
| [`secrets.tf`](terraform/secrets.tf) | CMEK-encrypted Secret Manager secret for the Socket API token |
| [`helm.tf`](terraform/helm.tf) | Namespace, K8s secret, Helm release (incl. pod `securityContext`/`fsGroup` so nginx can read the generated cert key) |
| [`helm.tf`](terraform/helm.tf) | Namespace, K8s secret, Helm release (fail-open + Redis stale cache + circuit breaker; pod `securityContext`/`fsGroup` so nginx can read the generated cert key) |
| [`redis.tf`](terraform/redis.tf) | Memorystore Redis (PSA, AUTH, TLS CA) + K8s secrets for the firewall verdict cache |
| [`tls.tf`](terraform/tls.tf) | Certificate Manager, SSL policy, GKE Gateway, GCPGatewayPolicy, HTTPRoute, HealthCheckPolicy |
| [`variables.tf`](terraform/variables.tf) | Input variable declarations (chart/image versions are required, no default) |
| [`terraform.tfvars`](terraform/terraform.tfvars) | Concrete pinned values Terraform auto-loads (project, SA emails, node counts, `firewall_domain`, chart/image versions) |
Expand Down Expand Up @@ -196,6 +224,8 @@ Roles Terraform grants to the **apply SA**:
| Custom `socketFirewallTfSecretManager` | Manage Secret Manager resources (no project-wide payload read) |
| `roles/secretmanager.secretAccessor` | Read the Socket API token (resource-scoped) |
| Custom `socketFirewallTfKmsManager` | CMEK key ring / key management (no key or version destruction) |
| `roles/redis.admin` | Memorystore Redis instance for the shared verdict cache |
| `roles/servicenetworking.networksAdmin` | Private Service Access peering for Memorystore |
| `roles/certificatemanager.editor` | Certificate Manager certs and DNS authorizations |
| `roles/iam.serviceAccountUser` | Attach the node SA to node-pool VMs |
| `roles/gkehub.gatewayEditor` + `roles/gkehub.viewer` | Reach the control plane via Connect Gateway; refresh the fleet membership |
Expand Down
1 change: 1 addition & 0 deletions terraform/apis.tf
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ locals {
"binaryauthorization.googleapis.com", # image admission policy enforcement
"gkehub.googleapis.com", # fleet membership for the cluster
"connectgateway.googleapis.com", # Connect Gateway proxy to the private control plane
"redis.googleapis.com", # Memorystore Redis for shared verdict cache
])
}

Expand Down
38 changes: 37 additions & 1 deletion terraform/helm.tf
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,44 @@ locals {
}
}

# failOpen=true: when the Socket API circuit breaker trips (or the API is
# unreachable), allow packages that have a cached verdict and fall back to
# allow for cache misses. Pair with Redis stale-while-revalidate below so
# the common case serves the last known-good decision instead of blocking.
# failOpenUnscanned stays false — unknown/unscanned packages are still blocked.
socket = {
existingSecret = kubernetes_secret.socket_api_token.metadata[0].name
existingSecretKey = "SOCKET_SECURITY_API_TOKEN"
failOpen = false
failOpen = true
failOpenUnscanned = false
# Fresh window for Socket API verdicts (seconds). After this, entries are
# stale and revalidated; Redis retains them until redis.ttl.
cacheTtl = 600
}

# Shared Redis cache (Memorystore). Fresh for cacheTtl, then stale until
# redis.ttl — on API/breaker failure the firewall serves the stale verdict.
redis = {
enabled = true
host = google_redis_instance.verdict_cache.host
port = 6378 # Memorystore TLS port
ttl = 86400
existingSecret = kubernetes_secret.redis_auth.metadata[0].name
existingSecretKey = "REDIS_PASSWORD"
ssl = true
sslVerify = true
sslServerName = google_redis_instance.verdict_cache.host
sslCaCertExistingSecret = kubernetes_secret.redis_ca.metadata[0].name
}

# Circuit breaker is not a first-class chart value yet; pass via the
# raw-config escape hatch (top-level socket.yml section).
extraConfig = {
resilience = {
circuit_breaker = {
enabled = true
}
}
}

pathRouting = local.firewall_domain != "" ? {
Expand Down Expand Up @@ -148,6 +181,9 @@ resource "helm_release" "socket_firewall" {
depends_on = [
kubernetes_namespace.socket_firewall,
kubernetes_secret.socket_api_token,
kubernetes_secret.redis_auth,
kubernetes_secret.redis_ca,
google_compute_firewall.allow_redis_egress,
]
}

Expand Down
21 changes: 21 additions & 0 deletions terraform/iam.tf
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,22 @@ resource "google_project_iam_member" "tf_sa_certificate_manager_editor" {
member = local.tf_sa_member
}

# Memorystore Redis instance lifecycle (verdict cache for the firewall)
resource "google_project_iam_member" "tf_sa_redis_admin" {
project = var.project_id
role = "roles/redis.admin"
member = local.tf_sa_member
}

# Private Service Access for Memorystore (allocate peering range + connection).
# compute.networkAdmin covers most of this; servicenetworking.networksAdmin is
# required for google_service_networking_connection.
resource "google_project_iam_member" "tf_sa_servicenetworking" {
project = var.project_id
role = "roles/servicenetworking.networksAdmin"
member = local.tf_sa_member
}

# ---------------------------------------------------------------------------
# Fleet Connect Gateway access for the apply SA
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -353,6 +369,11 @@ resource "google_project_iam_custom_role" "tf_plan_reader" {
# Fleet / GKE Hub
"gkehub.memberships.get",
"gkehub.memberships.list",
# Memorystore Redis
"redis.instances.get",
"redis.instances.list",
# Private Service Access connection used by Memorystore
"servicenetworking.services.get",
]
}

Expand Down
22 changes: 22 additions & 0 deletions terraform/network.tf
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,28 @@ resource "google_compute_firewall" "allow_egress" {
}
}

# Memorystore Redis with in-transit encryption listens on 6378 (not 6379).
# Scoped to the instance host so the broader deny-all egress still applies
# everywhere else.
resource "google_compute_firewall" "allow_redis_egress" {
name = "${var.cluster_name}-allow-redis-egress"
network = google_compute_network.main.id
direction = "EGRESS"
priority = 1000

allow {
protocol = "tcp"
ports = ["6378"]
}

destination_ranges = ["${google_redis_instance.verdict_cache.host}/32"]
target_tags = ["gke-${var.cluster_name}"]

log_config {
metadata = "INCLUDE_ALL_METADATA"
}
}

# Cloud NAT so private nodes can reach the internet
resource "google_compute_router" "main" {
name = "${var.cluster_name}-router"
Expand Down
10 changes: 10 additions & 0 deletions terraform/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,13 @@ output "firewall_domain" {
description = "Normalized domain used for path-based routing"
value = local.firewall_domain != "" ? local.firewall_domain : null
}

output "redis_host" {
description = "Memorystore Redis host for the shared Socket API verdict cache"
value = google_redis_instance.verdict_cache.host
}

output "redis_port" {
description = "Memorystore Redis TLS port (in-transit encryption)"
value = 6378
}
98 changes: 98 additions & 0 deletions terraform/redis.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# ---------------------------------------------------------------------------
# Memorystore Redis — shared Socket API verdict cache
# ---------------------------------------------------------------------------
# Multi-replica firewall pods need a shared cache so a tripped circuit breaker
# can serve the last known-good verdict (stale-while-revalidate) instead of
# blocking. In-cluster Redis would require Binary Authorization allowlisting
# of a Redis image; Memorystore avoids that and keeps Redis off the pod
# security boundary.
#
# Connectivity: PRIVATE_SERVICE_ACCESS on the firewall VPC. AUTH + in-transit
# TLS (server authentication) so the AUTH string and CA are mounted into the
# firewall pods via Kubernetes secrets.
# ---------------------------------------------------------------------------

resource "google_compute_global_address" "redis_psa" {
name = "${var.cluster_name}-redis-psa"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 24
network = google_compute_network.main.id
project = var.project_id

depends_on = [google_project_service.required]
}

resource "google_service_networking_connection" "redis_psa" {
network = google_compute_network.main.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.redis_psa.name]

depends_on = [google_project_service.required]
}

resource "google_redis_instance" "verdict_cache" {
name = "${var.cluster_name}-verdict-cache"
display_name = "Socket Firewall verdict cache"
tier = "BASIC"
memory_size_gb = var.redis_memory_size_gb
region = var.region
location_id = var.zone
redis_version = "REDIS_7_0"
authorized_network = google_compute_network.main.id
connect_mode = "PRIVATE_SERVICE_ACCESS"
auth_enabled = true
# SERVER_AUTHENTICATION enables TLS on port 6378 and exposes a per-instance
# CA that the firewall mounts via redis.sslCaCertExistingSecret.
transit_encryption_mode = "SERVER_AUTHENTICATION"

labels = {
app = "socket-firewall"
env = "prod"
team = "team-security-2"
}

depends_on = [
google_service_networking_connection.redis_psa,
google_project_service.required,
]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redis apply races IAM bindings

Medium Severity

PSA and Memorystore resources do not depend on the new roles/redis.admin and roles/servicenetworking.networksAdmin bindings for the apply SA. On the first apply of this change those resources can be created before the bindings land, so creation fails with permission errors and blocks the Redis plus fail-open rollout.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 9a90408. Configure here.

}

# AUTH string for the Helm chart (REDIS_PASSWORD env → redis-password secret).
resource "kubernetes_secret" "redis_auth" {
metadata {
name = "socket-firewall-redis-auth"
namespace = kubernetes_namespace.socket_firewall.metadata[0].name
}

data = {
REDIS_PASSWORD = google_redis_instance.verdict_cache.auth_string
}

type = "Opaque"

depends_on = [
kubernetes_namespace.socket_firewall,
google_redis_instance.verdict_cache,
]
}

# Memorystore in-transit encryption CA (private per-instance CA, not in the
# system trust store). Mounted by the chart at /etc/nginx/redis-tls/ca/ca.crt.
resource "kubernetes_secret" "redis_ca" {
metadata {
name = "socket-firewall-redis-ca"
namespace = kubernetes_namespace.socket_firewall.metadata[0].name
}

data = {
"ca.crt" = google_redis_instance.verdict_cache.server_ca_certs[0].cert
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CA secret ignores rotation certs

Medium Severity

The Redis CA secret stores only server_ca_certs[0]. During Memorystore CA rotation Google exposes multiple CAs and clients must trust all of them; keeping a single cert can make TLS verification fail, so the shared verdict cache drops out while failOpen is enabled.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 9a90408. Configure here.


type = "Opaque"

depends_on = [
kubernetes_namespace.socket_firewall,
google_redis_instance.verdict_cache,
]
}
6 changes: 6 additions & 0 deletions terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -162,3 +162,9 @@ variable "path_routing_routes" {
},
]
}

variable "redis_memory_size_gb" {
description = "Memorystore Redis memory size in GiB for the shared Socket API verdict cache"
type = number
default = 1
}
Loading