ADR-0017: Integration Testing Strategy

Status

Accepted

Context

While unit tests verify individual components in isolation, production issues often arise from:

Integration Points: Database queries, HTTP endpoints, middleware pipeline
Configuration: Settings applied only in running application
Infrastructure: EF Core migrations, database constraints, transactions
Request Pipeline: Authentication, authorization, middleware ordering
Serialization: JSON serialization/deserialization edge cases
Validation: End-to-end validation across layers

Unit tests alone cannot verify:

Actual HTTP request/response behavior
Database queries execute correctly with real EF provider
Configuration binds properly from appsettings.json
Middleware pipeline processes requests in correct order
Authentication/authorization work end-to-end
OpenAPI schema generation matches endpoint definitions

The application needed an integration testing strategy that:

Tests real HTTP endpoints with full request pipeline
Uses in-memory or test databases for isolation
Supports authentication scenarios
Provides fast feedback (seconds, not minutes)
Enables parallel execution without conflicts
Integrates with xUnit and existing test infrastructure

Decision

Adopt WebApplicationFactory<TProgram> for integration testing with test fixture pattern, in-memory authentication, test database per fixture, and endpoint-focused test organization.

Test Fixture Pattern

public class CustomWebApplicationFactoryFixture<TProgram> : WebApplicationFactory<TProgram>
    where TProgram : class
{
    private readonly EndpointTestFixtureOptions options;

    public CustomWebApplicationFactoryFixture(EndpointTestFixtureOptions options = null)
    {
        this.options = options ?? new EndpointTestFixtureOptions();
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseEnvironment("IntegrationTest");

        builder.ConfigureAppConfiguration((context, config) =>
        {
            // Override configuration for tests
            config.AddInMemoryCollection(new Dictionary<string, string>
            {
                ["ConnectionStrings:Default"] = "Server=(localdb)\\mssqllocaldb;Database=TestDb;Trusted_Connection=True;",
                ["Authentication:Enabled"] = "false" // Disable for most tests
            });
        });

        builder.ConfigureServices(services =>
        {
            // Replace services for testing
            services.RemoveAll<DbContextOptions<CoreDbContext>>();
            services.AddDbContext<CoreDbContext>(options =>
                options.UseInMemoryDatabase($"TestDb_{Guid.NewGuid()}"));
        });
    }
}

Endpoint Test Fixture with Authentication

public sealed class EndpointTestFixture : IAsyncLifetime
{
    private readonly CustomWebApplicationFactoryFixture<Program> factory;
    private readonly EndpointTestFixtureOptions options;
    private string bearerToken;

    public EndpointTestFixture(
        ITestOutputHelper output,
        EndpointTestFixtureOptions options = null)
    {
        this.Output = output;
        this.options = options ?? new EndpointTestFixtureOptions();
        this.factory = new CustomWebApplicationFactoryFixture<Program>(this.options);
    }

    public ITestOutputHelper Output { get; }

    public HttpClient CreateClient()
    {
        var client = this.factory.CreateClient();

        if (!string.IsNullOrEmpty(this.bearerToken))
        {
            client.DefaultRequestHeaders.Authorization =
                new AuthenticationHeaderValue("Bearer", this.bearerToken);
        }

        return client;
    }

    public async Task InitializeAsync()
    {
        if (this.options.UseAuthentication)
        {
            // Acquire JWT token for tests
            using var client = this.factory.CreateClient();
            var tokenRequest = new
            {
                grant_type = "password",
                client_id = this.options.ClientId,
                username = this.options.Username,
                password = this.options.Password,
                scope = this.options.Scope
            };

            var response = await client.PostAsJsonAsync(
                this.options.TokenEndpoint,
                tokenRequest);

            var tokenResponse = await response.Content.ReadFromJsonAsync<TokenResponse>();
            this.bearerToken = tokenResponse.AccessToken;
        }
    }

    public async Task DisposeAsync()
    {
        await this.factory.DisposeAsync();
    }
}

public class EndpointTestFixtureOptions
{
    public bool UseAuthentication { get; set; }
    public string TokenEndpoint { get; set; } = "/api/_system/fake-identity/token";
    public string ClientId { get; set; } = "test_client";
    public string Username { get; set; } = "test@example.com";
    public string Password { get; set; } = "Test123!";
    public string Scope { get; set; } = "api";
}

Integration Test Example

public class CustomerEndpointsTests : IClassFixture<EndpointTestFixture>
{
    private readonly EndpointTestFixture fixture;
    private readonly HttpClient client;

    public CustomerEndpointsTests(EndpointTestFixture fixture, ITestOutputHelper output)
    {
        this.fixture = new EndpointTestFixture(output);
        this.client = this.fixture.CreateClient();
    }

    [Fact]
    public async Task CreateCustomer_WithValidData_ReturnsCreatedWithLocation()
    {
        // Arrange
        var command = new CustomerCreateCommand("John Doe", "john@example.com");

        // Act
        var response = await this.client.PostAsJsonAsync("/api/core/customers", command);

        // Assert
        response.StatusCode.Should().Be(HttpStatusCode.Created);
        response.Headers.Location.Should().NotBeNull();

        var customerId = await response.Content.ReadFromJsonAsync<CustomerId>();
        customerId.Should().NotBeNull();
        customerId.Value.Should().NotBe(Guid.Empty);
    }

    [Fact]
    public async Task CreateCustomer_WithInvalidEmail_ReturnsBadRequest()
    {
        // Arrange
        var command = new CustomerCreateCommand("John Doe", "invalid-email");

        // Act
        var response = await this.client.PostAsJsonAsync("/api/core/customers", command);

        // Assert
        response.StatusCode.Should().Be(HttpStatusCode.BadRequest);

        var problemDetails = await response.Content.ReadFromJsonAsync<ValidationProblemDetails>();
        problemDetails.Errors.Should().ContainKey("Email");
    }

    [Fact]
    public async Task GetCustomers_ReturnsOkWithCustomers()
    {
        // Arrange - seed data
        var createCommand = new CustomerCreateCommand("Jane Doe", "jane@example.com");
        await this.client.PostAsJsonAsync("/api/core/customers", createCommand);

        // Act
        var response = await this.client.GetAsync("/api/core/customers");

        // Assert
        response.StatusCode.Should().Be(HttpStatusCode.OK);

        var customers = await response.Content.ReadFromJsonAsync<List<CustomerModel>>();
        customers.Should().NotBeEmpty();
        customers.Should().Contain(c => c.Email == "jane@example.com");
    }
}

Test Data Seeding

public static class TestDataFactory
{
    public static async Task<CustomerId> CreateCustomer(
        this HttpClient client,
        string name = "Test Customer",
        string email = "test@example.com")
    {
        var command = new CustomerCreateCommand(name, email);
        var response = await client.PostAsJsonAsync("/api/core/customers", command);
        response.EnsureSuccessStatusCode();

        return await response.Content.ReadFromJsonAsync<CustomerId>();
    }

    public static async Task<List<CustomerId>> CreateCustomers(
        this HttpClient client,
        int count)
    {
        var ids = new List<CustomerId>();
        for (int i = 0; i < count; i++)
        {
            var id = await client.CreateCustomer($"Customer {i}", $"customer{i}@example.com");
            ids.Add(id);
        }

        return ids;
    }
}

Rationale

Why WebApplicationFactory

Real Request Pipeline: Full ASP.NET Core pipeline including middleware, routing, model binding
Configuration Override: Easy to replace services and configuration for tests
In-Process: Runs in same process (fast, no network latency)
HttpClient: Tests use familiar HttpClient API
Isolation: Each test can have its own database/services
First-Class Support: Built into ASP.NET Core, well-documented

Why In-Memory Database

Speed: 10-100x faster than real database
Isolation: Each test fixture gets unique database
Cleanup: Database automatically disposed after tests
Parallelization: Tests run in parallel without conflicts
No Setup: No database server required
Deterministic: No external dependencies

Why Test Fixture Pattern

Setup/Teardown: IAsyncLifetime handles async initialization
Resource Sharing: Factory shared across test class
Authentication: Token acquired once per fixture
Test Output: ITestOutputHelper wired to Serilog sink
Configuration: Options pattern for flexible test setup

Why Endpoint-Focused Organization

Clear Scope: Tests organized by API endpoint
API Contract Verification: Tests verify OpenAPI-generated client behavior
User Perspective: Tests mimic actual API usage
Documentation: Tests serve as usage examples
Regression Detection: Breaking API changes caught immediately

Consequences

Positive

High Confidence: Tests verify actual HTTP behavior, not mocked interactions
Fast Feedback: In-memory database keeps tests fast (milliseconds per test)
Parallel Execution: Isolated databases enable parallel test runs
Realistic: Full request pipeline catches middleware, serialization issues
Documentation: Tests demonstrate actual API usage
Regression Prevention: Breaking changes to endpoints caught immediately
Authentication Testing: Fixture pattern simplifies auth scenarios
Easy Debugging: In-process execution simplifies debugging

Negative

Slower than Unit Tests: 10-100x slower than unit tests (still fast enough)
Setup Complexity: Factory and fixture setup more complex than unit tests
In-Memory Limitations: Some EF/database features not supported (stored procedures, triggers)
Data Seeding: Test data must be created via API or repository
Maintenance: Changes to Program.cs/Startup may break tests

Neutral

Test Database: In-memory vs. real database trade-off (speed vs. realism)
Authentication: Optional per fixture via EndpointTestFixtureOptions
Test Organization: One test class per endpoint or grouped by feature

Implementation Guidelines

Test Class Template

[Trait("Category", "Integration")]
public class MyEndpointsTests : IAsyncLifetime
{
    private readonly EndpointTestFixture fixture;
    private readonly HttpClient client;

    public MyEndpointsTests(ITestOutputHelper output)
    {
        this.fixture = new EndpointTestFixture(output, new EndpointTestFixtureOptions
        {
            UseAuthentication = true // Enable if endpoint requires auth
        });

        this.client = this.fixture.CreateClient();
    }

    public async Task InitializeAsync()
    {
        await this.fixture.InitializeAsync();

        // Seed test data
        await this.client.CreateCustomer();
    }

    public async Task DisposeAsync()
    {
        await this.fixture.DisposeAsync();
    }

    [Fact]
    public async Task Endpoint_Scenario_ExpectedResult()
    {
        // Arrange

        // Act

        // Assert
    }
}

Testing Validation Errors

[Fact]
public async Task CreateCustomer_WithMissingName_ReturnsBadRequest()
{
    // Arrange
    var command = new CustomerCreateCommand(null, "test@example.com");

    // Act
    var response = await this.client.PostAsJsonAsync("/api/core/customers", command);

    // Assert
    response.StatusCode.Should().Be(HttpStatusCode.BadRequest);

    var problemDetails = await response.Content.ReadFromJsonAsync<ValidationProblemDetails>();
    problemDetails.Should().NotBeNull();
    problemDetails.Errors.Should().ContainKey("Name");
}

Testing Filtering/Pagination

[Fact]
public async Task GetCustomers_WithFilter_ReturnsMatchingCustomers()
{
    // Arrange
    await this.client.CreateCustomer("Alice", "alice@example.com");
    await this.client.CreateCustomer("Bob", "bob@example.com");

    // Act
    var response = await this.client.GetAsync(
        "/api/core/customers?filter=Email:alice@example.com");

    // Assert
    response.StatusCode.Should().Be(HttpStatusCode.OK);

    var customers = await response.Content.ReadFromJsonAsync<List<CustomerModel>>();
    customers.Should().HaveCount(1);
    customers[0].Name.Should().Be("Alice");
}

Testing Authentication

[Fact]
public async Task GetCustomers_WithoutAuthentication_ReturnsUnauthorized()
{
    // Arrange
    var clientWithoutAuth = this.factory.CreateClient();
    // Don't add bearer token

    // Act
    var response = await clientWithoutAuth.GetAsync("/api/core/customers");

    // Assert
    response.StatusCode.Should().Be(HttpStatusCode.Unauthorized);
}

Using Real Database for Integration Tests

// When in-memory limitations are hit, use real database
builder.ConfigureServices(services =>
{
    services.RemoveAll<DbContextOptions<CoreDbContext>>();
    services.AddDbContext<CoreDbContext>(options =>
        options.UseSqlServer($"Server=(localdb)\\mssqllocaldb;Database=IntegrationTest_{Guid.NewGuid()};"));
});

// Cleanup in fixture disposal
public async Task DisposeAsync()
{
    // Drop test database
    await using var scope = this.factory.Services.CreateAsyncScope();
    var dbContext = scope.ServiceProvider.GetRequiredService<CoreDbContext>();
    await dbContext.Database.EnsureDeletedAsync();

    await this.factory.DisposeAsync();
}

Testing Background Jobs

[Fact]
public async Task CustomerExportJob_CreatesExportFile()
{
    // Arrange
    await this.client.CreateCustomers(5);

    // Trigger job manually
    var scheduler = this.factory.Services.GetRequiredService<IScheduler>();
    var job = new JobKey(nameof(CustomerExportJob));
    await scheduler.TriggerJob(job);

    // Wait for job completion
    await Task.Delay(TimeSpan.FromSeconds(2));

    // Assert
    // Verify export file or side effects
}

Alternatives Considered

Alternative 1: Manual Test Server Setup

var builder = WebApplication.CreateBuilder();
// Manual service registration
var app = builder.Build();
var server = new TestServer(app);

Rejected because:

More boilerplate than WebApplicationFactory
Doesn't leverage existing Program.cs
Harder to maintain as application evolves
WebApplicationFactory is standard ASP.NET Core pattern

Alternative 2: Postman/REST Client Tests

Rejected because:

Not automated in CI pipeline
No assertions or validation
Slower to run (external process)
No integration with xUnit test runner
Can't easily seed test data

Alternative 3: Separate Integration Test Project with Real Database

Rejected because:

Slower (database I/O)
Parallel execution difficult (shared database state)
Requires database server setup
More complex CI/CD configuration
In-memory sufficient for most scenarios

Alternative 4: BDD Framework (SpecFlow)

Rejected because:

Additional learning curve (Gherkin syntax)
Overkill for technical integration tests
xUnit provides sufficient readability with proper naming
SpecFlow better suited for acceptance tests with non-technical stakeholders

Related Decisions

ADR-0013: Integration tests complement unit tests
ADR-0014: Tests verify endpoint contracts
ADR-0016: ITestOutputHelper captures logs in tests
ADR-0018: Service replacement pattern in tests

References

Notes

Test Organization

tests/
  Modules/
    CoreModule/
      CoreModule.IntegrationTests/
        Presentation/
          Web/
            EndpointTestFixture.cs
            CustomerEndpointsTests.cs
            OrderEndpointsTests.cs
        Infrastructure/
          EntityFramework/
            CoreDbContextTests.cs  // Repository integration tests

Running Integration Tests

# Run all integration tests
dotnet test --filter "Category=Integration"

# Run specific test class
dotnet test --filter "FullyQualifiedName~CustomerEndpointsTests"

# Run with coverage
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

CI/CD Considerations

# GitHub Actions example
- name: Run Integration Tests
  run: dotnet test --filter "Category=IntegrationTest" --no-build --verbosity normal
  env:
    ASPNETCORE_ENVIRONMENT: IntegrationTest

Debugging Integration Tests

Set breakpoint in test or endpoint code
Debug test via Test Explorer or dotnet test
Full request pipeline executes in same process
Inspect ITestOutputHelper output for logs

Common Pitfalls

WRONG Sharing state between tests:

// WRONG - shared database leads to flaky tests
private static readonly CoreDbContext SharedDbContext;

CORRECT Isolate per fixture:

// CORRECT - each fixture gets unique in-memory database
services.AddDbContext<CoreDbContext>(options =>
    options.UseInMemoryDatabase($"TestDb_{Guid.NewGuid()}"));

WRONG Not disposing HttpClient:

// WRONG - resource leak
var client = this.factory.CreateClient();
// No disposal

CORRECT Use using statement:

// CORRECT
using var client = this.fixture.CreateClient();

Test Data Strategies

Inline Creation: Create data directly in test via API calls
Factory Methods: Reusable TestDataFactory extension methods
Fixture Seeding: Seed common data in InitializeAsync()
Test-Specific: Create unique data per test to avoid conflicts

FilesExpand file tree

0017-integration-testing-strategy.md

Latest commit

History

0017-integration-testing-strategy.md

File metadata and controls

ADR-0017: Integration Testing Strategy

Status

Context

Decision

Test Fixture Pattern

Endpoint Test Fixture with Authentication

Integration Test Example

Test Data Seeding

Rationale

Why WebApplicationFactory

Why In-Memory Database

Why Test Fixture Pattern

Why Endpoint-Focused Organization

Consequences

Positive

Negative

Neutral

Implementation Guidelines

Test Class Template

Testing Validation Errors

Testing Filtering/Pagination

Testing Authentication

Using Real Database for Integration Tests

Testing Background Jobs

Alternatives Considered

Alternative 1: Manual Test Server Setup

Alternative 2: Postman/REST Client Tests

Alternative 3: Separate Integration Test Project with Real Database

Alternative 4: BDD Framework (SpecFlow)

Related Decisions

References

Notes

Test Organization

Running Integration Tests

CI/CD Considerations

Debugging Integration Tests

Common Pitfalls

Test Data Strategies