Proposal: Add Assert.All<T> (aggregating per-element assertion) as an MSTest equivalent of xUnit's Assert.All

[Evangelink]

Summary

When migrating tests from xUnit to MSTest (e.g., dotnet/sdk#54732), there is no clean MSTest equivalent of xUnit''s Assert.All<T>(IEnumerable<T>, Action<T>). That xUnit API runs the inline assertion against every element of a collection and aggregates all failures into a single readable report (including the index of each failing item), instead of stopping at the first one.

The current MSTest workaround is to write a foreach loop:

// Before (xUnit)
Assert.All(group, e => e.Key!.ToString()!.StartsWith(prefix, StringComparison.Ordinal));

// After (MSTest, current)
foreach (DictionaryEntry entry in group)
{
    Assert.StartsWith(prefix, entry.Key!.ToString()!, StringComparison.Ordinal);
}

Real-world example from the migration: https://github.com/dotnet/sdk/pull/54732/files#diff-463b1612655ea283a5998dfc2988f2419af9f6e3ae5072d9ba5a6db2e359c71eL47-R51

Drawbacks of the workaround

• Fail-fast only — the test stops at the first bad element, hiding how widespread the issue is. Triage requires multiple rerun-and-fix cycles.
• No index/context in the failure message — the user has to instrument the loop manually (custom message, counter) to know which element failed.
• More noise in test code — a single semantic assertion turns into a multi-line block, which is friction during xUnit→MSTest migrations.

Proposal

Consider adding an Assert.All (name TBD) API along the lines of:

public static void All<T>(IEnumerable<T> collection, Action<T> assertion);
public static void All<T>(IEnumerable<T> collection, Action<T, int> assertion); // with index
public static Task AllAsync<T>(IEnumerable<T> collection, Func<T, Task> assertion);

Semantics (matching xUnit):

• Invokes assertion for every item.
• Catches AssertFailedException (and other exceptions) per item, continues iterating.
• Throws a single aggregated AssertFailedException listing each failing index, the offending item, and the inner failure message.

Open questions

1. Naming. Assert.All matches xUnit and reads naturally, but MSTest already uses Assert.* for single-value assertions. Alternatives: Assert.ForEach, CollectionAssert.All (though we are deprioritizing CollectionAssert.* — see the move toward modern Assert.* APIs).
2. Scope. Should this also have a Single/Contains(IEnumerable, predicate)-style sibling, or is All enough?
3. Analyzer support. Should MSTEST0037-style guidance suggest Assert.All over a hand-written foreach + Assert.*?

Context

• Originating discussion: dotnet/sdk PR #54732 (xUnit→MSTest.Sdk migration of Microsoft.NET.Build.Containers.UnitTests).
• xUnit reference: https://github.com/xunit/assert.xunit/blob/main/CollectionAsserts.cs (All, AllAsync).

[Evangelink]

Reviewed this against the bar I just applied to #9079 (closed as not planned). My take: keep this one open — it is a real capability gap, unlike Assert.Collection.

Why this is different from #9079

The killer feature is failure aggregation, not the name or shape. foreach (var x in xs) Assert.X(...) is fail-fast, so:

• You only learn about one violation per run → N rerun-and-fix cycles when a structural bug affects many items.
• You lose the pattern that often points to the real bug (e.g. "indices 3, 7, 12 fail" → "every 5th element is wrong").
• Index/context has to be manually plumbed via a counter + custom message.

Reproducing aggregation today requires ~8 lines of try/catch + list-of-failures + Assert.Fail(string.Join(...)) boilerplate at every call site. That cannot be hand-waved away the same way Assert.Collection could (since Assert.HasCount + indexed asserts already cover it).

Refinements I would make before implementing

1. Name. Assert.All is the lowest-friction migration path and matches xUnit, but All is a verb-less noun that does not read like an assertion. Assert.AllSatisfy (FluentAssertions naming) is more self-describing. Either is defensible — I lean All for migration ergonomics.
2. Which exceptions to catch. Spec must be explicit:
   • Catch AssertFailedException (and arguably Exception to match xUnit, so a broken inspector that throws an ArgumentException still aggregates).
   • Do not catch OperationCanceledException — cancellation should still short-circuit cleanly.
3. Indexed overload — yes: Action<T, int> removes the need for callers to do their own counting when they want the index in their own message.
4. No Func<T, bool> predicate overload. Skip it — it collides with Assert.IsTrue(items.All(p)) and discards diagnostic context. The lambda-with-asserts shape is strictly better.
5. Async overload — AllAsync<T>(IEnumerable<T>, Func<T, Task>). Worth considering IAsyncEnumerable<T> too.
6. Analyzer. A suggestion converting foreach (var x in xs) Assert.X(...) → Assert.All(xs, x => Assert.X(...)) is reasonable, but should be Info severity, not Warning — the aggregation behavior change is semantically meaningful enough that not every loop wants it.

Suggested final shape

public static partial class Assert
{
    public static void All<T>(IEnumerable<T> collection, Action<T> assertion);
    public static void All<T>(IEnumerable<T> collection, Action<T, int> assertion);
    public static Task AllAsync<T>(IEnumerable<T> collection, Func<T, Task> assertion);
    public static Task AllAsync<T>(IEnumerable<T> collection, Func<T, int, Task> assertion);
}

Failure message shape (sketch):

Assert.All failed. 3 out of 12 elements failed:
  [3]: <inner assertion message>
  [7]: <inner assertion message>
  [12]: <inner assertion message>

[Evangelink]

Reversing my earlier comment after @Evangelink pointed out that Assert.Scope() (Assert.Scope.cs, [Experimental("MSTESTEXP")]) already exists and provides the failure-aggregation primitive I called out as the killer feature.

using (Assert.Scope())
{
    foreach (var item in items)
        Assert.StartsWith(prefix, item.Key.ToString());
}

That:

• Collects every AssertFailedException from the body.
• Preserves each call site''s real stack trace via ExceptionDispatchInfo.Capture / Throw.
• Rethrows as a single AssertFailedException wrapping an AggregateException on dispose.

So aggregation is not a capability gap. My previous "~8 lines of try/catch boilerplate" claim was wrong — it''s 3 lines, and they reuse a first-class MSTest primitive.

What Assert.All would still add over Scope + foreach:

1. One line of sugar instead of three.
2. Auto-injected index in the failure message — but the caller can already write this manually via item / a counter in the inner assertion''s message argument.
3. Catching non-AssertFailedException like xUnit does — debatable whether this is even desirable; arguably a foot-gun.

What Scope does better than a hypothetical Assert.All:

• Works for any body shape, not just per-element loops (e.g. preconditions + multiple property checks on the same object).
• Composes with arbitrary control flow.

Net: the remaining argument for Assert.All is "swipe-in xUnit migration", which is the same bar that closed #9079. My updated recommendation: close as not planned and instead invest in:

• Stabilizing Assert.Scope (drop the [Experimental] once the API is settled — the MSTESTEXP diagnostic is the main reason people will not reach for it during migrations today).
• An xUnit→MSTest codefix that rewrites Assert.All(xs, x => …) into using (Assert.Scope()) { foreach (var x in xs) … }.

Sorry for the noise on the previous comment.

[Evangelink]

One more data point for triage: parameter-order convention also weakens the "swipe-in xUnit migration" argument.

For MSTest collection assertions that take a predicate-like companion, the convention is predicate/expected first, collection second:

• HasCount(int expected, IEnumerable collection, ...)
• Contains(T expected, IEnumerable<T> collection, ...)
• AreAllOfType(Type expectedType, IEnumerable collection, ...)
• ContainsSingle(Func<T,bool> predicate, IEnumerable<T> collection, ...)
• AreEqual(T expected, T actual, ...)

So the MSTest-aligned signature for the proposed API would be:

Assert.All<T>(Action<T> assertion, IEnumerable<T> collection, string? message = "")

…which is the reverse of xUnit''s Assert.All(collection, assertion). That means we can have either convention-consistency or mechanical migration ergonomics, not both. Combined with the fact that Assert.Scope() + foreach already provides the aggregation behavior, this is a third strike against shipping it.

Separately filed #9081 to rename Assert.Scope() -> Assert.BeginScope() before it leaves [Experimental].

[Evangelink]

Final recommendation: close as not planned; document the Assert.Scope + foreach pattern instead

After investigating the existing plumbing (Assert.cs:90-112 — ReportAssertFailed already routes failures through AssertScope.Current automatically) and weighing it against the FluentAssertions / xUnit precedent, I no longer think we should ship this API. Summary of the reasoning:

What Assert.Scope() + foreach already gives users today

using (Assert.Scope())
{
    foreach (var item in items)
    {
        Assert.StartsWith(prefix, item.Key.ToString());
    }
}

• Aggregation across all elements (this was the original "killer feature" of Assert.All).
• Per-element stack traces preserved via ExceptionDispatchInfo.Capture in AssertScope.AddError.
• Composition with any other assertions / control flow in the same test body.

What a hypothetical Assert.AllSatisfy(Action<T>, IEnumerable<T>) would uniquely add

Only one thing: per-element index automatically prepended to each captured failure message ("at index [3]: …"). Real but small — and the caller can add it manually via the inner assertion''s message argument or a counter when they want it.

What it would cost

• New public API on a deliberately tight surface.
• Forks the soft-assertion story (general Scope + specialized AllSatisfy) where MSTest currently has the cleaner single-primitive story (matching NUnit''s Assert.Multiple).
• Naive impl using (Assert.Scope()) { foreach … } inside the API would actually break if called inside a user''s outer Assert.Scope() (nesting is disallowed by AssertScope) and lose the per-element index (each inner exception carries only its own message). A correct impl would be a private try/catch loop calling ReportAssertFailed for the combined message — ~10 lines but still a new public API to maintain.

What to do instead

1. Document the canonical pattern in the xUnit / FluentAssertions migration guide: using (Assert.Scope()) { foreach (var x in xs) Assert.X(x); }. Note that Assert.Scope is currently [Experimental("MSTESTEXP")] — stabilizing it (separate issue, separate discussion) becomes a prerequisite for recommending this in migration docs without a warning suppression.
2. Codefix in the xUnit→MSTest analyzer that mechanically rewrites Assert.All(coll, inspector) → using (Assert.Scope()) { foreach (var x in coll) inspector(x); }. Same story for FluentAssertions'' coll.Should().AllSatisfy(inspector).

Cross-framework data point

Framework	Inspector-aggregating API	General soft-scope
xUnit	Assert.All(coll, inspector)	—
FluentAssertions	coll.Should().AllSatisfy(inspector)	using new AssertionScope()
NUnit	—	Assert.Multiple(Action)
MSTest	— (this issue)	using (Assert.Scope()) (Experimental)

xUnit and FA chose to ship a specialized inspector API on top of (or instead of) a general scope. NUnit didn''t and shipped only the scope. MSTest already has the scope. Joining the NUnit camp keeps the API surface lean and consistent.

Closing as not planned. Happy to reopen if the team disagrees.