Skip to content

04-layer-3-network-security.md

Latest commit

 

History

History
383 lines (318 loc) · 11.9 KB

04-layer-3-network-security.md

File metadata and controls

383 lines (318 loc) · 11.9 KB

Layer 3: Network Security

Architecture

The network security layer controls all outbound traffic from containers. Instead of allowing unrestricted internet access, a proxy validates each request using JWT tokens that specify allowed destinations. This prevents data exfiltration and command-and-control callbacks even if malware runs inside the container.

Network Stack Flow:

flowchart LR
    Container["Sandbox Container"] -->|"HTTP Request"| Proxy["Egress Proxy<br/>10.0.0.1:8443"]
    Proxy -->|"Validate JWT"| JWT["JWT Authentication"]
    JWT -->|"Check Host"| Validator["Host Validation<br/>(allowed_hosts)"]
    Validator -->|"Allowed"| Whitelist["Whitelisted Hosts"]
    Validator -->|"Blocked"| Deny["403 Forbidden"]
Loading

JWT-Based Network Control:

{
  "iss": "secure-egress-control",
  "container_id": "sandbox_session_xyz123",
  "allowed_hosts": "api.example.com,github.com,npmjs.com,pypi.org,...",
  "exp": 1700014400
}

Implementation Steps

The following steps deploy an Envoy proxy that acts as a gatekeeper for all container outbound traffic. The proxy validates JWTs and enforces host whitelisting.

1. Setup Egress Proxy (Envoy-based)

Install Envoy Proxy:

# Using Docker
docker pull envoyproxy/envoy:v1.28-latest

# Or install directly
curl -L https://getenvoy.io/cli | bash -s -- -b /usr/local/bin
getenvoy fetch envoy:v1.28.0

File: /etc/envoy/envoy.yaml

Production configuration with JWT authentication and Lua host filtering:

static_resources:
  listeners:
    - name: egress_listener
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 8443
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: egress_http
                route_config:
                  name: egress_route
                  virtual_hosts:
                    - name: egress_service
                      domains: ['*']
                      routes:
                        - match:
                            prefix: '/'
                          route:
                            cluster: dynamic_forward_proxy_cluster
                            timeout: 30s
                http_filters:
                  # JWT Authentication filter
                  - name: envoy.filters.http.jwt_authn
                    typed_config:
                      '@type': type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
                      providers:
                        sandbox_jwt:
                          issuer: 'secure-egress-control'
                          audiences:
                            - 'egress-proxy'
                          local_jwks:
                            inline_string: |
                              {
                                "keys": [
                                  {
                                    "kty": "EC",
                                    "crv": "P-256",
                                    "x": "replace-with-your-key-x",
                                    "y": "replace-with-your-key-y",
                                    "kid": "your-ecdsa-key-id"
                                  }
                                ]
                              }
                          forward: true
                          from_headers:
                            - name: 'Proxy-Authorization'
                              value_prefix: 'Basic '
                      rules:
                        - match:
                            prefix: '/'
                          requires:
                            provider_name: 'sandbox_jwt'
                  # Lua filter for host whitelist validation
                  - name: envoy.filters.http.lua
                    typed_config:
                      '@type': type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
                      inline_code: |
                        function envoy_on_request(request_handle)
                          local jwt_payload = request_handle:headers():get("x-jwt-payload")
                          if not jwt_payload then
                            request_handle:respond(
                              {[:status] = "403"},
                              "Missing JWT"
                            )
                            return
                          end

                          local json = require("cjson")
                          local payload = json.decode(jwt_payload)
                          local allowed_hosts = {}
                          for host in string.gmatch(payload.allowed_hosts, "[^,]+") do
                            allowed_hosts[host] = true
                          end

                          local target_host = request_handle:headers():get(":authority")
                          if not allowed_hosts[target_host] then
                            request_handle:respond(
                              {[:status] = "403", ["x-deny-reason"] = "host_not_allowed"},
                              "Host not in whitelist"
                            )
                            return
                          end
                        end
                  # Dynamic forward proxy enables outbound connections
                  - name: envoy.filters.http.dynamic_forward_proxy
                    typed_config:
                      '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
                      dns_cache_config:
                        name: dynamic_forward_proxy_cache_config
                        dns_lookup_family: V4_ONLY
                        max_hosts: 100
                  - name: envoy.filters.http.router
                    typed_config:
                      '@type': type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:
    - name: dynamic_forward_proxy_cluster
      lb_policy: CLUSTER_PROVIDED
      cluster_type:
        name: envoy.clusters.dynamic_forward_proxy
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
          dns_cache_config:
            name: dynamic_forward_proxy_cache_config
            dns_lookup_family: V4_ONLY
            max_hosts: 100

admin:
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 9901

The configuration above sets up JWT validation and Lua-based host filtering. The JWT filter validates tokens using the ES256 public key; the Lua script extracts allowed_hosts from the token payload and enforces the whitelist.

Start Envoy:

envoy -c /etc/envoy/envoy.yaml

2. Generate JWT Tokens

Containers need valid JWT tokens to make outbound network requests. The following Python script generates tokens with the container ID, user ID, and allowed hosts list embedded in the claims.

Python JWT Generator:

File: generate-egress-token.py

#!/usr/bin/env python3
# generate_egress_token.py

import jwt
import time
from datetime import datetime, timedelta

# EC private key (ES256)
PRIVATE_KEY = """
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIJ... (your private key)
-----END EC PRIVATE KEY-----
"""

def generate_egress_token(container_id, user_id, allowed_hosts, duration_hours=4):
    """Generate egress control JWT"""

    now = int(time.time())
    payload = {
        "iss": "secure-egress-control",
        "iat": now,
        "exp": now + (duration_hours * 3600),
        "container_id": container_id,
        "organization_uuid": user_id,
        "allowed_hosts": ",".join(allowed_hosts),
        "is_hipaa_regulated": "false",
        "use_egress_gateway": "true",
        "enforce_container_binding": "true", # RECOMMENDED: Set to true
        "enforce_centralized_egress": "false"
    }

    token = jwt.encode(
        payload,
        PRIVATE_KEY,
        algorithm="ES256",
        headers={"kid": "your-ecdsa-key-id"}
    )

    return token

# Example usage
if __name__ == "__main__":
    allowed_hosts = [
        "api.example.com",
        "github.com",
        "npmjs.com",
        "pypi.org",
        "archive.ubuntu.com"
    ]

    token = generate_egress_token(
        container_id="sandbox_session_abc123",
        user_id="user_123",
        allowed_hosts=allowed_hosts
    )

    print(f"JWT Token: {token}")

3. Container Network Configuration

Containers must not have direct internet access. Instead, route all traffic through the egress proxy using environment variables. The proxy validates JWTs and enforces host whitelisting on every request.

Disable default networking:

# Docker
docker run --runtime=runsc \
  --network=none \
  -e HTTP_PROXY="http://container_${CONTAINER_ID}:jwt_${TOKEN}@proxy:8443" \
  -e HTTPS_PROXY="http://container_${CONTAINER_ID}:jwt_${TOKEN}@proxy:8443" \
  -e NO_PROXY="localhost,127.0.0.1" \
  your-image

Kubernetes Network Policy:

NetworkPolicy is Kubernetes' built-in firewall. This policy blocks all egress traffic by default, then explicitly allows only DNS queries (to resolve hostnames) and traffic to the egress proxy (for validated outbound requests).

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: sandbox-egress-policy
spec:
  podSelector:
    matchLabels:
      app: sandbox
  policyTypes:
    - Egress
  egress:
    # Allow DNS
    - to:
        - namespaceSelector:
            matchLabels:
              name: kube-system
      ports:
        - protocol: UDP
          port: 53

    # Allow ONLY to egress proxy
    - to:
        - podSelector:
            matchLabels:
              app: egress-proxy
      ports:
        - protocol: TCP
          port: 8443

  # Block everything else (default deny)

4. Sandbox API Integration

File: sandbox_api.go

package main

import (
    "flag"
    "log"
    "net/http"
)

var (
    addr              = flag.String("addr", "0.0.0.0:8080", "Listen address")
    blockLocalConn    = flag.Bool("restrict-localhost", true, "Block localhost")
    maxWSBuffer       = flag.Int("websocket-buffer", 16384, "Max WebSocket buffer")
    memoryLimit       = flag.Int64("memory-limit-bytes", 4294900000, "Memory limit")
    cpuShares         = flag.Int("cpu-shares", 1024, "CPU shares")
)

func main() {
    flag.Parse()

    // Setup handlers
    http.HandleFunc("/execute", handleExecute)

    // Block localhost if configured
    if *blockLocalConn {
        http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
            if r.RemoteAddr == "127.0.0.1" || r.RemoteAddr == "::1" {
                w.WriteHeader(http.StatusForbidden)
                return
            }
        })
    }

    log.Printf("Starting process API on %s", *addr)
    log.Fatal(http.ListenAndServe(*addr, nil))
}

Allowed Hosts

The principle of least privilege applies to network access. Only allow the specific hosts your containers need to function. Start with a deny-all policy and explicitly whitelist necessary destinations like package managers and APIs.

Recommended Allowed Hosts:

# Package managers
package_managers:
  - npmjs.com
  - registry.npmjs.org
  - pypi.org
  - files.pythonhosted.org
  - crates.io
  - github.com

# OS updates
os_updates:
  - archive.ubuntu.com
  - security.ubuntu.com
  - deb.debian.org

# API access (customize per use case)
api_access:
  - api.your-service.com
  - your-cdn.cloudfront.net

# Block by default
blocked:
  - '*' # Deny all unless explicitly allowed

Testing Network Isolation

Verify that the egress proxy correctly blocks unauthorized destinations and allows whitelisted ones. These tests confirm that the JWT validation and host filtering are working properly.

# Should succeed (allowed)
curl http://api.example.com

# Should fail (not in whitelist)
curl http://evil-site.com
# Response: 403 Forbidden, x-deny-reason: host_not_allowed