Release 5.23.1

CHANGELOG.md

@@ -12,6 +12,66 @@
 
 -->
 
+# Relase 5.23 
+
+## release-notes
+
+* Changed: wire-server updated to version 5.23.0 for prod, wiab-staging and wiab-dev/demo
+* Changed: cargohold service will use the scoped `cargohold` user with least privilege, with access limited to its `assets` bucket only (#814)
+* Changed: Enable Ansible-based RabbitMQ deployment and fix RabbitMQ host configuration for wire-server (#861)
+
+### Data stores (PostgreSQL, Cassandra)
+
+* Added: enable support for PostgreSQL deployment via Ansible (#797)
+* Added: PostgreSQL high availability cluster with repmgr (#807)
+* Changed: PostgreSQL password management is now centralized in Kubernetes Secrets (repmgr and wire-server credentials), eliminating hardcoded passwords from inventory (#819)
+* Changed: update Cassandra from 3.11.16 to 3.11.19 (#831)
+
+### Features / configuration
+* Added: config for MLS deployment into example files (#824)
+
+## wire-builds
+
+* Changed: pre_clean_values_0.sh to clean unnecessary files
+  * Removed: `patch-chart-images.sh` as it is not required anymore
+  * Fixed: default|demo|min-build definitions to have more precise values and chart definitions (#825)
+* Changed: Standardized all scripts to use `yq-go` (v4+) for YAML processing, replacing deprecated `python-yq`. Updated syntax in offline deployment scripts (`cd.sh`, `cd-with-retry.sh`), build scripts (`build_adminhost_containers.sh`), demo deployment (`offline_deploy_k8s.sh`), secret sync utilities, and chart image extraction to ensure reliable YAML manipulation and fix CI build errors (#820)
+
+## deploy-builds
+
+### WIAB demo / staging (high‑level)
+
+* Fixed: coturn and PostgreSQL secrets for demo-wiab
+  * Added: `kube-prometheus-stack` values and enabled monitoring support from wire-server for demo-wiab
+  * Added: values for wire-utility in demo-wiab (#826)
+* Added: enable `cd-demo.sh` to verify demo-wiab builds (#826)
+* Changed: add Ansible playbook for wiab-staging VM provisioning
+  * Added: Terraform resources for wiab-staging
+  * Added: `cd_staging` script to verify the default build bundle
+  * Changed: restructured `offline.yml` flow – introduced wiab-staging build and split bundle processing with default-build (#861)
+
+### Offline / CI / deployment pipeline
+
+* Added: `bin/helm-operations.sh` to replace `offline-helm` and more closely follow production instructions
+  * Changed: `bin/offline-secrets.sh` to support `helm-operations.sh` and add support for coturn secret (#858)
+* Changed: Optimize Wire offline deployment pipeline with parallel job execution and S3 direct downloads
+  * Added: retry logic with progressive server type fallbacks for Hetzner Cloud resource availability issues (#815)
+* Changed: offline workflow to require explicit labels for PR builds (`build-default`, `build-demo`, `build-min`, `build-all`); PRs without labels no longer trigger builds (#836)
+* Changed: remove hardcoded PostgreSQL passwords from `demo-secrets.example.yaml` and automatically inject passwords from `databases-ephemeral` chart during deployment (#817)
+
+## docs
+
+* Added: documentation on how to set up DKIM for SMTP in wire-server (#793)
+* Added: enable cert-manager Helm chart deployment with example values files (#805)
+* Added: wiab-staging documentation to wire-server-deploy and fixed coturn port ranges (#861)
+* Added: Enable changelog management in wire-server-deploy (#764)
+
+## bug-fixes
+* Fixed: Optimize the `offline-env` load and add pipe/redirect functionality with `d` (#812)
+* Fixed:  add localhost authentication for `postgres_exporter`, upgrade to v0.18.1, and enable `stat_checkpointer` collector for PostgreSQL 17 checkpoint metrics (#832)
+* Fixed: changelog-verify.yml workflow to allow Zebot pushes to master (#806)
+* Changed: offline-vm-setup.sh script now uses an Ubuntu cloud image and local seed ISO (#861) 
+* Fixed: Update kubernetes_logging.yml to use the standard kubelet log path instead of Docker-specific paths. (#864)
 
 # 2021-08-27
 

ansible/inventory/demo/wiab-staging.yml

@@ -6,4 +6,4 @@ wiab-staging:
       ansible_user: 'demo'
       ansible_ssh_private_key_file: "~/.ssh/id_ed25519"
   vars:
-    artifact_hash: f1f624256bdab0f9f76158c7f45e0618ee641237
+    artifact_hash: 82edf88d9193e9f7e0a62ee4b287fd0c7cebb1bd

bin/debug_logs.sh

@@ -4,14 +4,14 @@ set -euo pipefail
 echo "Printing all pods status"
 kubectl get pods --all-namespaces
 echo "------------------------------------"
-namespaces=$(kubectl get ns -o=jsonpath='{.items[*].metadata.name}')
+namespaces="cert-manager-ns default"
 echo "Namespaces = $namespaces"
 for ns in $namespaces; do
-    pods=$(kubectl get pods --all-namespaces -o=jsonpath='{.items[*].metadata.name}')
+    pods=$(kubectl get pods -n "$ns" -o=jsonpath='{.items[*].metadata.name}')
     echo "Pods in namespace: $ns = $pods"
     for pod in $pods; do
         echo "Logs for pod: $pod"
-        kubectl logs --all-containers -n "$ns" "$pod" || true
+        kubectl logs --tail 30 --all-containers -n "$ns" "$pod" || true
         echo "Description for pod: $pod"
         kubectl describe pod -n "$ns" "$pod" || true
         echo "------------------------------------"

bin/helm-operations.sh

@@ -3,24 +3,57 @@
 set -Eeo pipefail
 
 # Read values from environment variables with defaults
-BASE_DIR="/wire-server-deploy"
-TARGET_SYSTEM="example.dev"
-CERT_MASTER_EMAIL="certmaster@${TARGET_SYSTEM}"
+BASE_DIR="${BASE_DIR:-/wire-server-deploy}"
+TARGET_SYSTEM="${TARGET_SYSTEM:-example.com}"
+CERT_MASTER_EMAIL="certmaster@${CERT_MASTER_EMAIL}:-certmaster@${TARGET_SYSTEM}"
+
+# DEPLOY_CERT_MANAGER env variable is used to decide if cert_manager and nginx-ingress-services charts should get deployed
+# default is set to TRUE to deploy it unless changed
+DEPLOY_CERT_MANAGER="${DEPLOY_CERT_MANAGER:-TRUE}"
+
+# DUMP_LOGS_ON_FAIL to dump logs on failure
+# it is false by default
+DUMP_LOGS_ON_FAIL="${DUMP_LOGS_ON_FAIL:-FALSE}"
 
 # this IP should match the DNS A record value for TARGET_SYSTEM
 # assuming it to be the public address used by clients to reach public Address 
-HOST_IP=""
+HOST_IP="${HOST_IP:-}"
+
 if [ -z "$HOST_IP" ]; then 
 HOST_IP=$(wget -qO- https://api.ipify.org)
 fi
 
+function dump_debug_logs {
+  local exit_code=$?
+  if [[ "$DUMP_LOGS_ON_FAIL" == "TRUE" ]]; then 
+    "$BASE_DIR"/bin/debug_logs.sh
+  fi
+  return $exit_code
+}
+trap dump_debug_logs ERR
+
 # picking a node for calling traffic (3rd kube worker node)
 CALLING_NODE=$(kubectl get nodes --no-headers | tail -n 1 | awk '{print $1}')
 if [[ -z "$CALLING_NODE" ]]; then
   echo "Error: could not determine the last kube worker node via kubectl"
   exit 1
 fi
 
+sync_pg_secrets() {
+  echo "Retrieving PostgreSQL password from databases-ephemeral for wire-server deployment..."
+  if kubectl get secret wire-postgresql-external-secret &>/dev/null; then
+  # Usage: sync-k8s-secret-to-wire-secrets.sh <secret-name> <secret-key> <yaml-file> <yaml-path's>
+     "$BASE_DIR/bin/sync-k8s-secret-to-wire-secrets.sh" \
+      wire-postgresql-external-secret password \
+      "$BASE_DIR/values/wire-server/secrets.yaml" \
+      .brig.secrets.pgPassword .galley.secrets.pgPassword
+  else
+    echo "⚠️  Warning: PostgreSQL secret 'wire-postgresql-secret' not found, skipping secret sync"
+    echo "    Make sure databases-ephemeral chart is deployed before wire-server"
+  fi
+  return $?
+}
+
 # Creates values.yaml from prod-values.example.yaml and secrets.yaml from prod-secrets.example.yaml
 # Works on all chart directories in $BASE_DIR/values/
 process_values() {
@@ -136,22 +169,6 @@ deploy_charts() {
       helm_command+=" --values $secrets_file"
     fi
 
-    # handle wire-server to inject PostgreSQL password from databases-ephemeral
-    if [[ "$chart" == "wire-server" ]]; then
-
-      echo "Retrieving PostgreSQL password from databases-ephemeral for wire-server deployment..."
-      if kubectl get secret wire-postgresql-secret &>/dev/null; then
-      # Usage: sync-k8s-secret-to-wire-secrets.sh <secret-name> <secret-key> <yaml-file> <yaml-path's>
-         "$BASE_DIR/bin/sync-k8s-secret-to-wire-secrets.sh" \
-          wire-postgresql-secret password \
-          "$BASE_DIR/values/wire-server/secrets.yaml" \
-          .brig.secrets.pgPassword .galley.secrets.pgPassword
-      else
-        echo "⚠️  Warning: PostgreSQL secret 'wire-postgresql-secret' not found, skipping secret sync"
-        echo "    Make sure databases-ephemeral chart is deployed before wire-server"
-      fi
-    fi
-
     echo "Deploying $chart as $helm_command"
     eval "$helm_command"
   done
@@ -163,7 +180,7 @@ deploy_charts() {
 deploy_cert_manager() {
 
   kubectl get namespace cert-manager-ns || kubectl create namespace cert-manager-ns
-  helm upgrade --install -n cert-manager-ns cert-manager  "$BASE_DIR/charts/cert-manager" --values "$BASE_DIR/values/cert-manager/values.yaml"
+  helm upgrade --install --wait --timeout=5m0s -n cert-manager-ns cert-manager  "$BASE_DIR/charts/cert-manager" --values "$BASE_DIR/values/cert-manager/values.yaml"
 
   # display running pods
   kubectl get pods --sort-by=.metadata.creationTimestamp -n cert-manager-ns
@@ -174,36 +191,45 @@ deploy_calling_services() {
   echo "Deploying sftd and coturn"
   # select the node to deploy sftd
   kubectl annotate node "$CALLING_NODE" wire.com/external-ip="$HOST_IP" --overwrite
-  helm upgrade --install sftd "$BASE_DIR/charts/sftd" --set "nodeSelector.kubernetes\\.io/hostname=$CALLING_NODE" --values  "$BASE_DIR/values/sftd/values.yaml"
+  helm upgrade --install --wait --timeout=5m0s sftd "$BASE_DIR/charts/sftd" --set "nodeSelector.kubernetes\\.io/hostname=$CALLING_NODE" --values  "$BASE_DIR/values/sftd/values.yaml"
 
   kubectl annotate node "$CALLING_NODE" wire.com/external-ip="$HOST_IP" --overwrite
-  helm upgrade --install coturn "$BASE_DIR/charts/coturn" --set "nodeSelector.kubernetes\\.io/hostname=$CALLING_NODE" --values "$BASE_DIR/values/coturn/values.yaml" --values "$BASE_DIR/values/coturn/secrets.yaml"
+  helm upgrade --install --wait --timeout=5m0s coturn "$BASE_DIR/charts/coturn" --set "nodeSelector.kubernetes\\.io/hostname=$CALLING_NODE" --values "$BASE_DIR/values/coturn/values.yaml" --values "$BASE_DIR/values/coturn/secrets.yaml"
+
+  # display running pods post deploying all helm charts in default namespace
+  kubectl get pods --sort-by=.metadata.creationTimestamp
 }
 
 main() {
+
 # Create prod-values.example.yaml to values.yaml and take backup
 process_values "prod" "values"
 # Create prod-secrets.example.yaml to secrets.yaml and take backup
 process_values "prod" "secrets"
 
+# Sync postgresql secret
+sync_pg_secrets
+
 # configure chart specific variables for each chart in values.yaml file
 configure_values
 
 # deploying with external datastores, useful for prod setup
 deploy_charts cassandra-external elasticsearch-external minio-external postgresql-external fake-aws smtp rabbitmq-external databases-ephemeral reaper wire-server webapp account-pages team-settings smallstep-accomp ingress-nginx-controller
 
-# deploying cert manager to issue certs, by default letsencrypt-http01 issuer is configured
-deploy_cert_manager
+# deploying cert-manager only when the env var DEPLOY_CERT_MANAGER is set to TRUE 
+if [[ "$DEPLOY_CERT_MANAGER" == "TRUE" ]]; then 
+  # deploying cert manager to issue certs, by default letsencrypt-http01 issuer is configured
+  deploy_cert_manager
 
-# nginx-ingress-services chart needs cert-manager to be deployed
-deploy_charts nginx-ingress-services
+  # nginx-ingress-services chart needs cert-manager to be deployed
+  deploy_charts nginx-ingress-services
+
+  # print status of certs
+  kubectl get certificate
+fi
 
 # deploying sft and coturn services
-# not implemented yet
 deploy_calling_services
-
-# print status of certs
-kubectl get certificate
 }
 
 main

bin/offline-deploy.sh

@@ -41,15 +41,4 @@ fi
 
 $DOCKER_RUN_BASE $SSH_MOUNT $WSD_CONTAINER ./bin/offline-cluster.sh
 
-# Sync PostgreSQL password from K8s secret to secrets.yaml
-echo "Syncing PostgreSQL password from Kubernetes secret..."
-sudo docker run --network=host -v $PWD:/wire-server-deploy $WSD_CONTAINER ./bin/sync-k8s-secret-to-wire-secrets.sh \
-  wire-postgresql-external-secret \
-  password \
-  values/wire-server/prod-secrets.example.yaml \
-  .brig.secrets.pgPassword \
-  .galley.secrets.pgPassword \
-  .spar.secrets.pgPassword \
-  .gundeck.secrets.pgPassword
-
-sudo docker run --network=host -v $PWD:/wire-server-deploy $WSD_CONTAINER ./bin/helm-operations.sh
+sudo docker run --network=host -v $PWD:/wire-server-deploy $WSD_CONTAINER sh -c 'TARGET_SYSTEM="example.dev" CERT_MASTER_EMAIL="certmaster@example.dev" DEPLOY_CERT_MANAGER=TRUE DUMP_LOGS_ON_FAIL=TRUE ./bin/helm-operations.sh'

bin/offline-secrets.sh

@@ -53,12 +53,7 @@ brig:
       password: guest
     # These are only necessary if you wish to support sign up via SMS/calls
     # And require accounts at twilio.com / nexmo.com
-    setTwilio: |-
-      sid: "dummy"
-      token: "dummy"
-    setNexmo: |-
-      key: "dummy"
-      secret: "dummy"
+
 cargohold:
   secrets:
     awsKeyId: "$minio_cargohold_access_key"

bin/secrets.sh

@@ -104,14 +104,7 @@ brig:
         # these only need to be changed if using real AWS services
         awsKeyId: dummykey
         awsSecretKey: dummysecret
-        # These are only necessary if you wish to support sign up via SMS/calls
-        # And require accounts at twilio.com / nexmo.com
-        setTwilio: |-
-            sid: dummy
-            token: dummy
-        setNexmo: |-
-            key: dummy
-            secret: dummy
+
 cargohold:
     secrets:
         awsKeyId: dummykey

changelog.d/0-release-notes/release-5.23

@@ -0,0 +1 @@
+Removed: removing all old changelogs to cut a release for wire-server-deploy 5.23 for prod, wiab-staging and wiab-dev 

changelog.d/3-deploy-builds/minor-deploy-fixes

@@ -0,0 +1,8 @@
+Fixed: debug_logs.sh to log only the pods for default and cert-manager-ns namespace and limit log lines
+Added: enabled debug_logs.sh on helm install failures (helm_operations.sh) with a flag DUMP_LOGS_ON_FAIL
+Added: env vars to helm_operations.sh to improve UX while configuring variables 
+Fixed: sync_pg_secrets operation in helm_operations.sh and clean the deploy_charts logic 
+Added: wait and timeout on cert-manager and calling_services helm chart operations
+Fixed: offline-cluster.sh to run helm-operations.sh using new env vars and with default DUMP_LOGS_ON_FAIL=TRUE 
+Fixed: documentation for wiab-staging.md based on a user feedback
+Fixed: sftd helm chart values for joinCall component which fails to find hashbased images

changelog.d/3-deploy-builds/remove-oboslete-secrets

@@ -0,0 +1 @@
+Removed: obsolete secrets from scripts and references for improved security and clarity.