Skip to content

Add rolling-updates feature flag and compatibility framework #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akshayutture-augment wants to merge 1 commit into
base: feature-rolling-updates-baseline
Choose a base branch
from
Open

Add rolling-updates feature flag and compatibility framework #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion common/src/main/java/org/keycloak/common/Profile.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,9 @@ public enum Feature {

USER_EVENT_METRICS("Collect metrics based on user events", Type.PREVIEW),

IPA_TUURA_FEDERATION("IPA-Tuura user federation provider", Type.EXPERIMENTAL)
IPA_TUURA_FEDERATION("IPA-Tuura user federation provider", Type.EXPERIMENTAL),

ROLLING_UPDATES("Rolling Updates", Type.PREVIEW),
;

private final Type type;
Expand Down
15 changes: 12 additions & 3 deletions docs/guides/operator/advanced-configuration.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,12 @@ Check the https://kubernetes.io/docs/concepts/services-networking/network-polici

The Keycloak Operator offers updates strategies to control how the Operator handles changes to the Keycloak CR.

[CAUTION]
====
While on preview stage, the feature `rolling-updates` must be enabled.
Otherwise, the {project_name} Operator will fail.
====

**Supported Updates Types:**

Rolling Updates:: Update the StatefulSet in a rolling fashion, minimizing downtime (requires multiple replicas).
Expand All @@ -466,11 +472,14 @@ kind: Keycloak
metadata:
name: example-kc
spec:
features:
enabled:
- rolling-updates # <1>
update:
strategy: Recreate|<not set> # <1>
strategy: Recreate|<not set> # <2>
----

<1> Set the desired update strategy here (Recreate in this example).
<1> Enable preview feature `rolling-updates`.
<2> Set the desired update strategy here (Recreate in this example).

[%autowidth]
.Possible field values
Expand Down
10 changes: 9 additions & 1 deletion docs/guides/server/update-compatibility.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,11 @@ preview="true"
previewDiscussionLink="https://github.com/keycloak/keycloak/discussions/36785"
>

// TODO Link to discussion?
[CAUTION]
====
While on preview stage, the feature `rolling-updates` must be enabled.
Otherwise, the commands will fail.
====

The goal of this tool is to assist with modifying a {project_name} deployment, whether upgrading to a new version, enabling/disabling features, or changing configuration.
The outcome will indicate whether a rolling upgrade is possible or if a recreate upgrade is required.
Expand Down Expand Up @@ -124,6 +128,10 @@ m|2
m|3
|Rolling Upgrade is not possible.
The deployment must be shut down before applying the new configuration.

m|4
|Rolling Upgrade is not possible.
The feature `rolling-updates` is disabled.
|===

</@tmpl.guide>
2 changes: 1 addition & 1 deletion docs/guides/templates/kc.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@ bin/kc.[sh|bat] bootstrap-admin ${parameters}
<#macro updatecompatibility parameters>
[source,bash]
----
bin/kc.[sh|bat] update-compatibility ${parameters}
bin/kc.[sh|bat] update-compatibility ${parameters} --features=rolling-updates
----
</#macro>
2 changes: 1 addition & 1 deletion operator/scripts/Dockerfile-custom-image
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ ARG IMAGE=keycloak
ARG VERSION=latest
FROM $IMAGE:$VERSION

RUN /opt/keycloak/bin/kc.sh build --db=postgres --health-enabled=true
RUN /opt/keycloak/bin/kc.sh build --db=postgres --health-enabled=true --features=rolling-updates
11 changes: 7 additions & 4 deletions operator/src/test/java/org/keycloak/operator/testsuite/integration/UpgradeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

package org.keycloak.operator.testsuite.integration;

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
Expand All @@ -27,10 +28,11 @@
import org.awaitility.Awaitility;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import org.keycloak.common.Profile;
import org.keycloak.operator.crds.v2alpha1.deployment.Keycloak;
import org.keycloak.operator.crds.v2alpha1.deployment.KeycloakStatusCondition;
import org.keycloak.operator.crds.v2alpha1.deployment.ValueOrSecret;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.UnsupportedSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.FeatureSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.UpdateSpec;
import org.keycloak.operator.upgrade.UpdateStrategy;

Expand Down Expand Up @@ -105,11 +107,12 @@ private static Keycloak createInitialDeployment(UpdateStrategy updateStrategy) {
}
var updateSpec = new UpdateSpec();
updateSpec.setStrategy(updateStrategy);
kc.getSpec().setUpdateSpec(updateSpec);

if (kc.getSpec().getUnsupported() == null) {
kc.getSpec().setUnsupported(new UnsupportedSpec());
if (kc.getSpec().getFeatureSpec() == null) {
kc.getSpec().setFeatureSpec(new FeatureSpec());
}
kc.getSpec().setUpdateSpec(updateSpec);
kc.getSpec().getFeatureSpec().setEnabledFeatures(List.of(Profile.Feature.ROLLING_UPDATES.getKey()));
return kc;
}

Expand Down
4 changes: 4 additions & 0 deletions ...untime/src/main/java/org/keycloak/quarkus/runtime/cli/command/AbstractUpdatesCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,8 @@ void printPreviewWarning() {
printError("Warning! This command is preview and is not recommended for use in production. It may change or be removed at a future release.");
}

void printFeatureDisabled() {
printError("Unable to use this command. The preview feature 'rolling-updates' is not enabled.");
}

}
7 changes: 7 additions & 0 deletions ...time/src/main/java/org/keycloak/quarkus/runtime/cli/command/UpdateCompatibilityCheck.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
import java.io.File;
import java.io.IOException;

import org.keycloak.common.Profile;
import org.keycloak.quarkus.runtime.cli.PropertyException;
import org.keycloak.quarkus.runtime.compatibility.CompatibilityResult;
import org.keycloak.quarkus.runtime.compatibility.ServerInfo;
import org.keycloak.util.JsonSerialization;
import picocli.CommandLine;
Expand All @@ -41,6 +43,11 @@ public class UpdateCompatibilityCheck extends AbstractUpdatesCommand {

@Override
public void run() {
if (!Profile.isFeatureEnabled(Profile.Feature.ROLLING_UPDATES)) {
printFeatureDisabled();
picocli.exit(CompatibilityResult.FEATURE_DISABLED);
return;
}
printPreviewWarning();
validateConfig();
var info = readServerInfo();
Expand Down
7 changes: 7 additions & 0 deletions ...e/src/main/java/org/keycloak/quarkus/runtime/cli/command/UpdateCompatibilityMetadata.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@
import java.io.IOException;

import com.fasterxml.jackson.core.JsonProcessingException;
import org.keycloak.common.Profile;
import org.keycloak.quarkus.runtime.cli.PropertyException;
import org.keycloak.quarkus.runtime.compatibility.CompatibilityResult;
import org.keycloak.quarkus.runtime.compatibility.ServerInfo;
import org.keycloak.util.JsonSerialization;
import picocli.CommandLine;
Expand All @@ -41,6 +43,11 @@ public class UpdateCompatibilityMetadata extends AbstractUpdatesCommand {

@Override
public void run() {
if (!Profile.isFeatureEnabled(Profile.Feature.ROLLING_UPDATES)) {
printFeatureDisabled();
picocli.exit(CompatibilityResult.FEATURE_DISABLED);
return;
}
printPreviewWarning();
validateConfig();
var info = compatibilityManager.current();
Expand Down
6 changes: 5 additions & 1 deletion ...runtime/src/main/java/org/keycloak/quarkus/runtime/compatibility/CompatibilityResult.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@
public interface CompatibilityResult {

int ROLLING_UPGRADE_EXIT_CODE = 0;
int RECREATE_UPGRADE_EXIT_CODE = 4;
// see picocli.CommandLine.ExitCode
// 1 -> software error
// 2 -> usage error
int RECREATE_UPGRADE_EXIT_CODE = 3;
int FEATURE_DISABLED = 4;
Comment on lines +31 to +35

@coderabbitai coderabbitai Bot Nov 22, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

Exit code changes introduce breaking change.

The restructuring of exit codes introduces a breaking change:

  • RECREATE_UPGRADE_EXIT_CODE changed from 4 → 3
  • New FEATURE_DISABLED = 4

While the alignment with picocli conventions (1=software error, 2=usage error) is good, any existing automation or scripts that rely on exit code 4 meaning "recreate upgrade required" will break.

Recommendations:

  1. Document this breaking change in release notes
  2. Consider providing a migration guide for automation/scripts
  3. Verify that internal Keycloak tooling and operator tests have been updated

Run the following script to check for any existing tests or scripts that assert on exit code 4 in the context of recreate upgrades:

🏁 Script executed:

#!/bin/bash
# Search for test assertions or scripts checking exit code 4
rg -n -C3 'exit.*4|exitCode.*4|assertEquals.*4|assertThat.*4' --type java --type sh -g '!CompatibilityResult.java' -g '!update-compatibility.adoc'

Length of output: 285197

Breaking change confirmed; document and manage migration impact.

The script search found no existing test assertions on exit code 4 in the codebase, but this does not invalidate the breaking change identified. The code clearly shows the exit code reassignment: RECREATE_UPGRADE_EXIT_CODE changes from 4 to 3, with 4 now assigned to FEATURE_DISABLED. This is a confirmed breaking change.

The absence of existing tests checking exit code 4 suggests the change either lacks test coverage or the relevant tests exist in external repositories (operator, Keycloak tooling, deployment scripts). This strengthens the need for the original review recommendations:

  1. Document the breaking change in release notes
  2. Provide a migration guide for users relying on exit code 4 for recreate upgrade logic
  3. Verify external tooling (Keycloak operator, deployment automation) that may depend on exit code semantics

The code change itself appears intentional and correct. However, without evidence that this breaking change is documented or that downstream dependencies have been updated, this requires management and communication.

🤖 Prompt for AI Agents 
In
quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/compatibility/CompatibilityResult.java
around lines 31-35, the exit code constants were reordered so
RECREATE_UPGRADE_EXIT_CODE is now 3 and 4 is FEATURE_DISABLED, which is a
confirmed breaking change; update project artifacts to manage this by (1) adding
a clear entry in the release notes documenting the exit code reassignment and
its impact, (2) creating a migration guide that shows mapping from old to new
exit codes and recommended changes for scripts/operators that relied on exit
code 4, (3) auditing and updating downstream tooling (operator, CI/deploy
scripts, automation) that may interpret exit codes, and (4) adding or updating
tests that assert the expected exit codes (including a regression test verifying
the new values) to prevent silent regressions.


/**
* The compatible {@link CompatibilityResult} implementation
Expand Down
20 changes: 14 additions & 6 deletions quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/UpdateCommandDistTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,36 +41,44 @@
@RawDistOnly(reason = "Requires creating JSON file to be available between containers")
public class UpdateCommandDistTest {

private static final String ENABLE_FEATURE = "--features=rolling-updates";

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME})
public void testFeatureNotEnabled(CLIResult cliResult) {
cliResult.assertError("Unable to use this command. The preview feature 'rolling-updates' is not enabled.");
}

@Test
@Launch({UpdateCompatibility.NAME})
public void testMissingSubCommand(CLIResult cliResult) {
cliResult.assertError("Missing required subcommand");
}

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME})
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, ENABLE_FEATURE})
public void testMissingOptionOnSave(CLIResult cliResult) {
cliResult.assertNoMessage("Missing required argument");
}

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME})
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, ENABLE_FEATURE})
public void testMissingOptionOnCheck(CLIResult cliResult) {
cliResult.assertError("Missing required argument: " + UpdateCompatibilityCheck.INPUT_OPTION_NAME);
}

@Test
public void testCompatible(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible");
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, UpdateCompatibilityMetadata.OUTPUT_OPTION_NAME, jsonFile.getAbsolutePath());
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, UpdateCompatibilityMetadata.OUTPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertMessage("Metadata:");
assertEquals(0, result.exitCode());

var info = JsonSerialization.mapper.readValue(jsonFile, ServerInfo.class);
assertEquals(Version.VERSION, info.getVersions().get(CompatibilityManagerImpl.KEYCLOAK_VERSION_KEY));
assertEquals(org.infinispan.commons.util.Version.getVersion(), info.getVersions().get(CompatibilityManagerImpl.INFINISPAN_VERSION_KEY));

result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertMessage("[OK] Rolling Upgrade is available.");
result.assertNoError("Rolling Upgrade is not available.");
}
Expand All @@ -85,7 +93,7 @@ public void testWrongVersions(KeycloakDistribution distribution) throws IOExcept
CompatibilityManagerImpl.INFINISPAN_VERSION_KEY, org.infinispan.commons.util.Version.getVersion()));
JsonSerialization.mapper.writeValue(jsonFile, info);

var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertError("[Versions] Rolling Upgrade is not available. 'keycloak' is incompatible: Old=0.0.0.Final, New=%s".formatted(Version.VERSION));

// incompatible infinispan version
Expand All @@ -94,7 +102,7 @@ public void testWrongVersions(KeycloakDistribution distribution) throws IOExcept
CompatibilityManagerImpl.INFINISPAN_VERSION_KEY, "0.0.0.Final"));
JsonSerialization.mapper.writeValue(jsonFile, info);

result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertError("[Versions] Rolling Upgrade is not available. 'infinispan' is incompatible: Old=0.0.0.Final, New=%s".formatted(org.infinispan.commons.util.Version.getVersion()));
}

Expand Down