scale up the resources - #13
Merged
Merged
Conversation
|
Terraform plan in terraform Plan: 1 to add, 3 to change, 0 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
!~ update in-place
<= read (data resources)
Terraform will perform the following actions:
# data.kubernetes_service.socket_firewall will be read during apply
# (depends on a resource or a module with changes pending)
<= data "kubernetes_service" "socket_firewall" {
+ id = (known after apply)
+ spec = (known after apply)
+ status = (known after apply)
+ metadata {
+ generation = (known after apply)
+ name = "socket-firewall"
+ namespace = "socket-firewall"
+ resource_version = (known after apply)
+ uid = (known after apply)
}
}
# google_container_node_pool.main will be updated in-place
!~ resource "google_container_node_pool" "main" {
id = "projects/sentry-socket/locations/us-central1-a/clusters/socket-firewall/nodePools/socket-firewall-nodes"
name = "socket-firewall-nodes"
# (11 unchanged attributes hidden)
!~ autoscaling {
!~ max_node_count = 3 -> 6
# (4 unchanged attributes hidden)
}
# (4 unchanged blocks hidden)
}
# helm_release.socket_firewall will be updated in-place
!~ resource "helm_release" "socket_firewall" {
id = "socket-firewall"
!~ metadata = [
- {
- app_version = "2.1.1"
- chart = "socket-firewall"
- first_deployed = 1781303659
- last_deployed = 1786573577
- name = "socket-firewall"
- namespace = "socket-firewall"
- notes = <<-EOT
Socket Firewall deployed successfully!
## Self-Signed Certificates
Self-signed certificates were generated. Extract the CA certificate:
kubectl exec -n socket-firewall \
$(kubectl get pod -n socket-firewall -l app.kubernetes.io/name=socket-firewall -o jsonpath='{.items[0].metadata.name}') \
-- cat /etc/nginx/ssl/ca.crt > socket-ca.crt
Then trust the CA on developer machines or use a real certificate for production.
## Accessing the Firewall
For testing, use port-forward:
kubectl port-forward svc/socket-firewall 8443:443 -n socket-firewall
For production, expose via Ingress or change service.type to LoadBalancer.
## Usage
Path-based routing is enabled on domain: sfw.security.sentry.io
Configure package managers to use path prefixes:
npm: https://sfw.security.sentry.io/npm/
pypi: https://sfw.security.sentry.io/pypi/
maven: https://sfw.security.sentry.io/maven/
Example:
npm config set registry https://sfw.security.sentry.io/npm/
pip config set global.index-url https://sfw.security.sentry.io/pypi/simple
## Verify
Test the health endpoint:
curl -sk https://<firewall-ip>/health
## Metrics
Prometheus metrics are exposed on port 9145 at /metrics (plain HTTP, no auth).
Set metrics.serviceMonitor.enabled=true (Prometheus Operator) or metrics.podAnnotations=true
(annotation-based) to scrape it. The endpoint is unauthenticated — restrict access with a NetworkPolicy.
See deployment recommendations in README.md for production setup.
EOT
- revision = 19
- values = jsonencode(
{
- affinity = {
- podAntiAffinity = {
- preferredDuringSchedulingIgnoredDuringExecution = [
- {
- podAffinityTerm = {
- labelSelector = {
- matchLabels = {
- "app.kubernetes.io/instance" = "socket-firewall"
}
}
- topologyKey = "kubernetes.io/hostname"
}
- weight = 100
},
]
}
}
- autoscaling = {
- enabled = true
}
- extraConfig = {
- resilience = {
- circuit_breaker = {
- enabled = true
}
}
}
- image = {
- pullPolicy = "IfNotPresent"
- repository = "socketdev/socket-registry-firewall"
- tag = "2.1.1"
}
- initContainers = {
- certGenerator = {
- securityContext = {
- allowPrivilegeEscalation = false
- capabilities = {
- drop = [
- "ALL",
]
}
- readOnlyRootFilesystem = true
- runAsNonRoot = true
- runAsUser = 1001
- seccompProfile = {
- type = "RuntimeDefault"
}
}
}
}
- pathRouting = {
- domain = "sfw.security.sentry.io"
- enabled = true
- routes = [
- {
- path = "/npm"
- registry = "npm"
- upstream = "https://registry.npmjs.org"
},
- {
- path = "/pypi"
- registry = "pypi"
- upstream = "https://pypi.org"
},
- {
- path = "/maven"
- registry = "maven"
- upstream = "https://repo1.maven.org/maven2"
},
]
}
- podSecurityContext = {
- fsGroup = 1001
}
- redis = {
- enabled = true
- existingSecret = "socket-firewall-redis-auth"
- existingSecretKey = "REDIS_PASSWORD"
- host = "10.236.196.99"
- port = 6378
- ssl = true
- sslCaCertExistingSecret = "socket-firewall-redis-ca"
- sslServerName = "10.236.196.99"
- sslVerify = true
- ttl = 86400
}
- replicaCount = 2
- resources = {
- limits = {
- cpu = "1"
- memory = "768Mi"
}
- requests = {
- cpu = "500m"
- memory = "512Mi"
}
}
- service = {
- httpsTargetPort = "http"
- type = "ClusterIP"
}
- socket = {
- cacheTtl = 600
- existingSecret = "socket-api-token"
- existingSecretKey = "SOCKET_SECURITY_API_TOKEN"
- failOpen = true
- failOpenUnscanned = false
}
- tls = {
- generateSelfSigned = true
}
}
)
- version = "0.11.2"
},
] -> (known after apply)
name = "socket-firewall"
!~ values = [
!~ <<-EOT
"affinity":
"podAntiAffinity":
"preferredDuringSchedulingIgnoredDuringExecution":
- "podAffinityTerm":
"labelSelector":
"matchLabels":
"app.kubernetes.io/instance": "socket-firewall"
"topologyKey": "kubernetes.io/hostname"
"weight": 100
"autoscaling":
"enabled": true
+ "maxReplicas": 6
+ "minReplicas": 2
+ "targetCPUUtilizationPercentage": 60
"extraConfig":
"resilience":
"circuit_breaker":
"enabled": true
"image":
"pullPolicy": "IfNotPresent"
"repository": "socketdev/socket-registry-firewall"
"tag": "2.1.1"
"initContainers":
"certGenerator":
"securityContext":
"allowPrivilegeEscalation": false
"capabilities":
"drop":
- "ALL"
"readOnlyRootFilesystem": true
"runAsNonRoot": true
"runAsUser": 1001
"seccompProfile":
"type": "RuntimeDefault"
+ "keepaliveTimeout": 620
+ "nginx":
+ "workerProcesses": 2
"pathRouting":
"domain": "sfw.security.sentry.io"
"enabled": true
"routes":
- "path": "/npm"
"registry": "npm"
"upstream": "https://registry.npmjs.org"
- "path": "/pypi"
"registry": "pypi"
"upstream": "https://pypi.org"
- "path": "/maven"
"registry": "maven"
"upstream": "https://repo1.maven.org/maven2"
"podSecurityContext":
"fsGroup": 1001
"redis":
"enabled": true
"existingSecret": "socket-firewall-redis-auth"
"existingSecretKey": "REDIS_PASSWORD"
"host": "10.236.196.99"
"port": 6378
"ssl": true
"sslCaCertExistingSecret": "socket-firewall-redis-ca"
"sslServerName": "10.236.196.99"
"sslVerify": true
"ttl": 86400
"replicaCount": 2
"resources":
"limits":
- "cpu": "1"
- "memory": "768Mi"
+ "cpu": "1500m"
+ "memory": "3Gi"
"requests":
- "cpu": "500m"
- "memory": "512Mi"
+ "cpu": "750m"
+ "memory": "3Gi"
"service":
"httpsTargetPort": "http"
"type": "ClusterIP"
"socket":
"cacheTtl": 600
"existingSecret": "socket-api-token"
"existingSecretKey": "SOCKET_SECURITY_API_TOKEN"
"failOpen": true
"failOpenUnscanned": false
+ "terminationGracePeriodSeconds": 210
"tls":
"generateSelfSigned": true
EOT,
]
# (26 unchanged attributes hidden)
}
# kubectl_manifest.socket_firewall_backend_policy[0] will be created
+ resource "kubectl_manifest" "socket_firewall_backend_policy" {
+ api_version = "networking.gke.io/v1"
+ apply_only = false
+ field_manager = "kubectl"
+ force_conflicts = false
+ force_new = false
+ id = (known after apply)
+ kind = "GCPBackendPolicy"
+ live_manifest_incluster = (sensitive value)
+ live_uid = (known after apply)
+ name = "socket-firewall-backend-policy"
+ namespace = "socket-firewall"
+ server_side_apply = false
+ uid = (known after apply)
+ upgrade_api_version = false
+ validate_schema = true
+ wait_for_rollout = true
+ yaml_body = (sensitive value)
+ yaml_body_parsed = <<-EOT
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: socket-firewall-backend-policy
namespace: socket-firewall
spec:
default:
connectionDraining:
drainingTimeoutSec: 60
logging:
enabled: true
timeoutSec: 300
targetRef:
group: ""
kind: Service
name: socket-firewall
EOT
+ yaml_incluster = (sensitive value)
}
# kubectl_manifest.socket_firewall_health_check[0] will be updated in-place
!~ resource "kubectl_manifest" "socket_firewall_health_check" {
id = "/apis/networking.gke.io/v1/namespaces/socket-firewall/healthcheckpolicys/socket-firewall-healthcheck"
name = "socket-firewall-healthcheck"
!~ yaml_body = (sensitive value)
!~ yaml_body_parsed = <<-EOT
apiVersion: networking.gke.io/v1
kind: HealthCheckPolicy
metadata:
name: socket-firewall-healthcheck
namespace: socket-firewall
spec:
default:
+ checkIntervalSec: 5
config:
httpHealthCheck:
portSpecification: USE_SERVING_PORT
requestPath: /health
type: HTTP
+ healthyThreshold: 1
+ timeoutSec: 5
+ unhealthyThreshold: 3
targetRef:
group: ""
kind: Service
name: socket-firewall
EOT
# (15 unchanged attributes hidden)
}
Plan: 1 to add, 3 to change, 0 to destroy.✅ Plan applied in Terraform Apply #25 Outputscluster_name = "socket-firewall"
firewall_domain = "sfw.security.sentry.io"
firewall_health_url = "https://sfw.security.sentry.io/health"
firewall_load_balancer_ip = "34.98.104.179"
firewall_namespace = "socket-firewall"
firewall_service_name = "socket-firewall"
kubeconfig_command = "gcloud container fleet memberships get-credentials socket-firewall --project sentry-socket"
redis_host = "10.236.196.99"
redis_port = 6378
tls_certificate_name = "socket-firewall-cert"
tls_dns_authorization_record = {
data = "6a407a91-86a0-4dc8-a434-e72cdc012e0e.9.authorize.certificatemanager.goog."
name = "_acme-challenge.sfw.security.sentry.io."
type = "CNAME"
} |
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Comment on lines
+34
to
+44
|
|
||
| # Google's guidance for the same error on GKE NEG backends: | ||
| # Tgrace > TpreStop + Tdrain + Tbuffer, at least 210s. Tdrain is the 60s | ||
| # connection draining in tls.tf. | ||
| # | ||
| # TpreStop is 0 here because the chart exposes no preStop hook, so nothing | ||
| # holds the pod open for the ~120s the endpoint can take to leave the NEG. | ||
| # In-flight requests are covered, but requests the LB sends in that window | ||
| # still land on a shutting-down pod. That residue is the reason to expect | ||
| # this error to shrink rather than vanish. | ||
| terminationGracePeriodSeconds = 210 |
There was a problem hiding this comment.
Bug: The keepaliveTimeout and terminationGracePeriodSeconds values may be ignored by the Helm chart if they are not supported as top-level keys, causing a silent failure.
Severity: HIGH
Suggested Fix
Verify the values.yaml of the socket-firewall Helm chart (v0.11.2) to confirm the correct structure for configuring keepaliveTimeout and terminationGracePeriodSeconds. These values might need to be nested under a specific key rather than being at the top level. Adjust the local.helm_values map in helm.tf to match the chart's expected structure.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: terraform/helm.tf#L33-L44
Potential issue: The Terraform configuration sets `keepaliveTimeout` and
`terminationGracePeriodSeconds` as top-level values for the `socket-firewall` Helm
chart. It is unverified if the chart (v0.11.2) supports these keys at the top level. If
it does not, Helm will silently ignore them. This would cause the pod's termination
grace period to default to 30s instead of the intended 210s, which is shorter than the
60s connection draining timeout, potentially dropping in-flight requests. Similarly, the
`keepaliveTimeout` would remain at its default of 65s instead of 620s, failing to
mitigate the `backend_connection_closed_before_data_sent_to_client` issue this PR aims
to fix.
Did we get this right? 👍 / 👎 to inform future reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.