[WIP] Migrates Azure storage tests to use Testcontainers by Meir017 · Pull Request #9921 · dotnet/orleans

Meir017 · 2026-02-16T09:33:30Z

related to #9708

Microsoft Reviewers: Open in CodeFlow

…Storage - Add AzuriteContainer static singleton that lazily starts an Azurite Testcontainer (or reuses an already-running instance on default ports). - Replace StorageEmulator usage in TestUtils.CheckForAzureStorage() with AzuriteContainer.EnsureStartedAsync(). - Fall back DataConnectionString to AzuriteContainer.ConnectionString when no explicit config is provided. - Add Testcontainers package reference to TestExtensions project.

Azurite is now managed in-process by Testcontainers via AzuriteContainer. The Event Hubs job retains the Azurite service container because the Event Hubs emulator Docker container depends on it for blob/metadata storage, but its ORLEANSDATACONNECTIONSTRING is removed since the test process will detect the already-running Azurite on default ports.

…r setup Replace manual ContainerBuilder configuration with AzuriteBuilder from the Testcontainers.Azurite NuGet package. This provides built-in image selection, port bindings, wait strategies, and a GetConnectionString() method. Also enables WithInMemoryPersistence() for faster test execution.

Resolves naming conflict with Testcontainers.Azurite.AzuriteContainer.

Remove the AAD authentication support from test infrastructure since all Azure Storage tests now use the Azurite emulator via Testcontainers. - Remove UseAadAuthentication, TokenCredential, TableEndpoint, DataBlobUri, DataQueueUri, and EventHubFullyQualifiedNamespace from TestDefaultConfiguration - Simplify CheckForAzureStorage to only start the Azurite container - Simplify CheckForEventHub to only check connection string - Remove AAD branches from AzureStorageOperationOptionsExtensions, EventHubConfigurationExtensions, CosmosOptionsExtensions, JournalingAzureStorageTestConfiguration, and other test files - Remove unused Azure.Identity package from TestExtensions.csproj

…tring Remove the redundant DataConnectionString property from TestDefaultConfiguration and use AzuriteContainerManager.ConnectionString directly in all test files.

Copilot

Pull request overview

This is a work-in-progress PR that migrates Azure Storage testing infrastructure from GitHub Actions services to Testcontainers, specifically focusing on Azurite. The change is part of a broader initiative (issue #9708) to simplify the testing infrastructure and make tests easier to run locally.

Changes:

Introduces AzuriteContainerManager to manage a singleton Azurite container using Testcontainers
Removes Azure AD authentication support from test configuration (TestDefaultConfiguration)
Updates all Azure Storage test configurations to use the new container manager instead of environment variables
Removes Azurite service definition from GitHub Actions workflow for Azure Storage tests

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/TestInfrastructure/TestExtensions/AzuriteContainerManager.cs	New singleton manager for Azurite Testcontainer with lazy initialization
test/TestInfrastructure/TestExtensions/TestDefaultConfiguration.cs	Removed Azure AD authentication properties and connection string configurations
test/TestInfrastructure/TestExtensions/TestUtils.cs	Updated CheckForAzureStorage to use AzuriteContainerManager
test/TestInfrastructure/TestExtensions/TestExtensions.csproj	Replaced Azure.Identity dependency with Testcontainers.Azurite
test/TesterInternal/GeoClusterTests/BasicLogTestGrainTests.cs	Migrated to use AzuriteContainerManager connection string
test/Tester/Forwarding/ShutdownSiloTests.cs	Migrated to use AzuriteContainerManager connection string
test/Extensions/TesterAzureUtils/Streaming/TestAzureTableStorageStreamFailureHandler.cs	Migrated to use AzuriteContainerManager connection string
test/Extensions/TesterAzureUtils/Streaming/AQStreamingTests.cs	Migrated to use AzuriteContainerManager connection string
test/Extensions/TesterAzureUtils/StorageEmulatorUtilities.cs	Updated to check AzuriteContainerManager connection string
test/Extensions/TesterAzureUtils/AzureStorageOperationOptionsExtensions.cs	Removed Azure AD auth logic and migrated all extension methods to use AzuriteContainerManager
test/Extensions/Tester.Cosmos/CosmosOptionsExtensions.cs	Simplified to remove Azure AD authentication support
test/Extensions/ServiceBus.Tests/EventHubConfigurationExtensions.cs	Simplified to remove Azure AD authentication support
test/Orleans.Journaling.Tests/JournalingAzureStorageTestConfiguration.cs	Updated to use AzuriteContainerManager connection string
test/Benchmarks/Transactions/TransactionBenchmark.cs	Migrated to use AzuriteContainerManager connection string
test/Benchmarks/GrainStorage/GrainStorageBenchmark.cs	Migrated to use AzuriteContainerManager connection string (with bug in AdoNet configurator)
NuGet.Config	Commented out Azure DevOps package feeds (should not be in this PR)
src/Directory.Build.props	Commented out Microsoft.DotNet.GenAPI.Task (should not be in this PR)
Directory.Packages.props	Added Testcontainers.Azurite package version
.github/workflows/ci.yml	Removed Azurite service and connection string from Azure Storage test job

test/Benchmarks/GrainStorage/GrainStorageBenchmark.cs

test/TestInfrastructure/TestExtensions/AzuriteContainerManager.cs

.github/workflows/ci.yml

test/TestInfrastructure/TestExtensions/AzuriteContainerManager.cs

test/Orleans.Journaling.Tests/JournalingAzureStorageTestConfiguration.cs

…tils Use a Lazy<bool> for thread-safe singleton initialization instead of manual lock and Task caching. Add synchronous EnsureStarted() with SkipException, matching the ConsulTestUtils pattern.

…initialization

…v var Tests using StandaloneSiloHandle spawn silos in separate processes. Each process was starting its own Azurite container, so silos registered gateways in a different container than the one the client queried. Propagate the connection string via ORLEANS_AZURITE_CONNECTION_STRING environment variable so child processes reuse the same container.

Meir017 · 2026-02-16T12:50:37Z

getting errors Docker API responded with status code=NotFound, response={"message":"No such image: mcr.microsoft.com/azure-storage/azurite:3.35.0"}

seems random 😔

ReubenBond · 2026-02-16T17:41:33Z

test/Benchmarks/GrainStorage/GrainStorageBenchmark.cs

            hostBuilder.AddAzureTableGrainStorageAsDefault(options =>
            {
-                options.TableServiceClient = new(TestDefaultConfiguration.DataConnectionString);
+                options.TableServiceClient = new(AzuriteContainerManager.ConnectionString);


When we run these tests in internal Azure DevOps, we target real storage backends and TestDefaultConfiguration loads real creds. It looks like this would hard-code the tests to always use Azurite

@ReubenBond oh, so we want to keep using these dedicated real storage accounts? for some reason I assumed we want to use the testcontainers everywhere...

@Meir017 We want Testcontainers in GitHub (since we don't want to expose secrets, etc) but we need the option to test against real services, too, to improve confidence since sometimes there are gaps (Cosmos DB is a good example of gaps between emulator and real service).

ReubenBond · 2026-02-17T00:20:00Z

@Meir017 the Consul Testcontainers commit is causing failures in our internal Azure DevOps, I suppose we aren't allowed to access docker.io from there.

Docker.DotNet.DockerApiException : Docker API responded with status code=InternalServerError, response={"message":"Get "https://registry-1.docker.io/v2/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response) in /_/src/Docker.DotNet/DockerClient.cs:line 516
   at Docker.DotNet.DockerClient.MakeRequestForRawResponseAsync(HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, CancellationToken token) in /_/src/Docker.DotNet/DockerClient.cs:line 331
   at Docker.DotNet.Models.StreamUtil.MonitorResponseForMessagesAsync[T](Task`1 responseTask, DockerClient client, CancellationToken cancel, IProgress`1 progress) in /_/src/Docker.DotNet/Endpoints/StreamUtil.cs:line 80
   at DotNet.Testcontainers.Clients.DockerImageOperations.CreateAsync(IImage image, IDockerRegistryAuthenticationConfiguration dockerRegistryAuthConfig, CancellationToken ct) in /_/src/Testcontainers/Clients/DockerImageOperations.cs:line 73
   at DotNet.Testcontainers.Clients.TestcontainersClient.PullImageAsync(IImage image, CancellationToken ct) in /_/src/Testcontainers/Clients/TestcontainersClient.cs:line 430
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(IContainerConfiguration configuration, CancellationToken ct) in /_/src/Testcontainers/Clients/TestcontainersClient.cs:line 337
   at DotNet.Testcontainers.Containers.DockerContainer.UnsafeCreateAsync(CancellationToken ct) in /_/src/Testcontainers/Containers/DockerContainer.cs:line 493
   at DotNet.Testcontainers.Containers.DockerContainer.StartAsync(CancellationToken ct) in /_/src/Testcontainers/Containers/DockerContainer.cs:line 337
   at Consul.Tests.ConsulTestUtils.EnsureConsulAsync() in /_/test/Extensions/Consul.Tests/ConsulTestUtils.cs:line 42
   at Consul.Tests.ConsulMembershipTableTest.GetConnectionString() in /_/test/Extensions/Consul.Tests/ConsulMembershipTableTest.cs:line 77
   at UnitTests.ConnectionStringFixture.get_ConnectionString() in /_/test/TesterInternal/ConnectionStringFixture.cs:line 25
   at UnitTests.MembershipTests.MembershipTableTestsBase..ctor(ConnectionStringFixture fixture, TestEnvironmentFixture environment, LoggerFilterOptions filters) in /_/test/TesterInternal/MembershipTests/MembershipTableTestsBase.cs:line 54
   at Consul.Tests.ConsulMembershipTableTest..ctor(ConnectionStringFixture fixture, TestEnvironmentFixture environment) in /_/test/Extensions/Consul.Tests/ConsulMembershipTableTest.cs:line 31
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Constructor(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)

I'm trying to see if we can fix that (#9933). We might need to disable Testcontainers tests in AzDO.

Meir017 · 2026-02-17T04:33:11Z

@Meir017 the Consul Testcontainers commit is causing failures in our internal Azure DevOps, I suppose we aren't allowed to access docker.io from there.

Docker.DotNet.DockerApiException : Docker API responded with status code=InternalServerError, response={"message":"Get "https://registry-1.docker.io/v2/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response) in /_/src/Docker.DotNet/DockerClient.cs:line 516
   at Docker.DotNet.DockerClient.MakeRequestForRawResponseAsync(HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, CancellationToken token) in /_/src/Docker.DotNet/DockerClient.cs:line 331
   at Docker.DotNet.Models.StreamUtil.MonitorResponseForMessagesAsync[T](Task`1 responseTask, DockerClient client, CancellationToken cancel, IProgress`1 progress) in /_/src/Docker.DotNet/Endpoints/StreamUtil.cs:line 80
   at DotNet.Testcontainers.Clients.DockerImageOperations.CreateAsync(IImage image, IDockerRegistryAuthenticationConfiguration dockerRegistryAuthConfig, CancellationToken ct) in /_/src/Testcontainers/Clients/DockerImageOperations.cs:line 73
   at DotNet.Testcontainers.Clients.TestcontainersClient.PullImageAsync(IImage image, CancellationToken ct) in /_/src/Testcontainers/Clients/TestcontainersClient.cs:line 430
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(IContainerConfiguration configuration, CancellationToken ct) in /_/src/Testcontainers/Clients/TestcontainersClient.cs:line 337
   at DotNet.Testcontainers.Containers.DockerContainer.UnsafeCreateAsync(CancellationToken ct) in /_/src/Testcontainers/Containers/DockerContainer.cs:line 493
   at DotNet.Testcontainers.Containers.DockerContainer.StartAsync(CancellationToken ct) in /_/src/Testcontainers/Containers/DockerContainer.cs:line 337
   at Consul.Tests.ConsulTestUtils.EnsureConsulAsync() in /_/test/Extensions/Consul.Tests/ConsulTestUtils.cs:line 42
   at Consul.Tests.ConsulMembershipTableTest.GetConnectionString() in /_/test/Extensions/Consul.Tests/ConsulMembershipTableTest.cs:line 77
   at UnitTests.ConnectionStringFixture.get_ConnectionString() in /_/test/TesterInternal/ConnectionStringFixture.cs:line 25
   at UnitTests.MembershipTests.MembershipTableTestsBase..ctor(ConnectionStringFixture fixture, TestEnvironmentFixture environment, LoggerFilterOptions filters) in /_/test/TesterInternal/MembershipTests/MembershipTableTestsBase.cs:line 54
   at Consul.Tests.ConsulMembershipTableTest..ctor(ConnectionStringFixture fixture, TestEnvironmentFixture environment) in /_/test/Extensions/Consul.Tests/ConsulMembershipTableTest.cs:line 31
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Constructor(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)

I'm trying to see if we can fix that (#9933). We might need to disable Testcontainers tests in AzDO.

I believe they have some strict rate limits on dockerhub. So creating a custom container registry and copying specific images could work (like the aspire repo does)

Meir017 added 6 commits February 16, 2026 10:34

Rename AzuriteContainer to AzuriteContainerManager

8d43906

Resolves naming conflict with Testcontainers.Azurite.AzuriteContainer.

Replace DataConnectionString with AzuriteContainerManager.ConnectionS…

7c4c8c2

…tring Remove the redundant DataConnectionString property from TestDefaultConfiguration and use AzuriteContainerManager.ConnectionString directly in all test files.

Copilot AI review requested due to automatic review settings February 16, 2026 09:33

Copilot started reviewing on behalf of Meir017 February 16, 2026 09:34 View session

remove local hacks

ac5d9c0

Meir017 mentioned this pull request Feb 16, 2026

Simplify testing infrastructure to use Testcontainers #9708

Open

13 tasks

Meir017 changed the title ~~[WIP] Feature/azurite testcontainers~~ [WIP] Migrates Azure storage tests to use Testcontainers Feb 16, 2026

Copilot AI reviewed Feb 16, 2026

View reviewed changes

Meir017 added 5 commits February 16, 2026 11:42

nit

45a2b73

Refactor AzuriteContainerManager to use Lazy pattern like ConsulTestU…

5f95a22

…tils Use a Lazy<bool> for thread-safe singleton initialization instead of manual lock and Task caching. Add synchronous EnsureStarted() with SkipException, matching the ConsulTestUtils pattern.

Update AzuriteContainerManager to specify Azurite image in container …

d08e7c2

…initialization

run azure storage tests on dedicated workflow

32c086b

ReubenBond reviewed Feb 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Migrates Azure storage tests to use Testcontainers#9921

[WIP] Migrates Azure storage tests to use Testcontainers#9921
Meir017 wants to merge 12 commits intodotnet:mainfrom
Meir017:feature/azurite-testcontainers

Meir017 commented Feb 16, 2026 •

edited by dotnet-policy-service bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Meir017 commented Feb 16, 2026

Uh oh!

ReubenBond Feb 16, 2026

Uh oh!

Meir017 Feb 16, 2026

Uh oh!

ReubenBond Feb 16, 2026 •

edited

Loading

Uh oh!

ReubenBond commented Feb 17, 2026 •

edited

Loading

Uh oh!

Meir017 commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Meir017 commented Feb 16, 2026 • edited by dotnet-policy-service bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Microsoft Reviewers: Open in CodeFlow

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Meir017 commented Feb 16, 2026

Uh oh!

ReubenBond Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Meir017 Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

ReubenBond Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ReubenBond commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Meir017 commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Meir017 commented Feb 16, 2026 •

edited by dotnet-policy-service bot

Loading

ReubenBond Feb 16, 2026 •

edited

Loading

ReubenBond commented Feb 17, 2026 •

edited

Loading