Skip to content

[Test] Switching to testing with Swift Testing#2118

Merged
cachemeifyoucan merged 3 commits into
swiftlang:mainfrom
cachemeifyoucan:eng/PR-switch-to-swift-testing
May 5, 2026
Merged

[Test] Switching to testing with Swift Testing#2118
cachemeifyoucan merged 3 commits into
swiftlang:mainfrom
cachemeifyoucan:eng/PR-switch-to-swift-testing

Conversation

@cachemeifyoucan
Copy link
Copy Markdown
Contributor

@cachemeifyoucan cachemeifyoucan commented Apr 13, 2026

Switching to Swift Testing.

The big SwiftDriverTests has been split up and the split is following:

DriverOptionTests.swift (29 tests)

  • invocationRunModes
  • subcommandsHandling
  • driverKindParsing
  • compilerMode
  • joinedPathOptions
  • relativeOptionOrdering
  • runtimeCompatibilityVersion
  • inputFiles
  • dashE
  • dashEJoined
  • primaryOutputKinds
  • ltoOutputModeClash
  • ltoOutputPath
  • primaryOutputKindsDiagnostics
  • filePrefixMapInvalidDiagnostic
  • filePrefixMapMultiplePassToFrontend
  • indexIncludeLocals
  • debugSettings
  • coverageSettings
  • hermeticSealAtLink
  • abiDescriptorOnlyWhenEnableEvolution
  • moduleSettings
  • moduleNameFallbacks
  • packageNameFlag
  • moduleABIName
  • allowableClient
  • publicModuleName
  • moduleNaming
  • duplicateName

CompileJobTests.swift (35 tests)

  • standardCompileJobs
  • emitModuleSeparatelyDiagnosticPath
  • emitModuleSeparatelyDependenciesPath
  • batchModeCompiles
  • batchModeDiagnostics
  • singleThreadedWholeModuleOptimizationCompiles
  • multiThreadedWholeModuleOptimizationCompiles
  • wholeModuleOptimizationOutputFileMap
  • wholeModuleOptimizationUsingSupplementaryOutputFileMap
  • wmoWithNonSourceInput
  • explicitBuildWithJustObjectInputs
  • wmoWithNonSourceInputFirstAndModuleOutput
  • wmoWithJustObjectInputs
  • emitModuleSeparately
  • emitModuleSeparatelyWMO
  • moduleWrapJob
  • multithreading
  • multithreadingDiagnostics
  • multiThreadingOutputs
  • pchGeneration
  • pcmGeneration
  • pcmDump
  • indexFilePathHandling
  • indexMultipleFilesInSingleCommandLineInvocation
  • indexFileEntryInSupplementaryFileOutputMap
  • pchAsCompileInput
  • internalPCHasCompileInput
  • cxxInteropOptions
  • embeddedSwiftOptions
  • dashDashPassingDownInput
  • dashDashImmediateInput
  • emitModuleEmittingDependencies
  • emitConstValues
  • emitModuleSepratelyEmittingDiagnosticsWithOutputFileMap
  • emitPCHWithOutputFileMap

LinkJobTests.swift (15 tests)

  • linking
  • lEqualPassedDownToLinkerInvocation
  • compatibilityLibs
  • dsymGeneration
  • verifyDebugInfo
  • emitModuleTrace
  • cxxLinking
  • ltoOption
  • ltoOutputs
  • ltoLibraryArg
  • bcAsTopLevelOutput
  • gccToolchainFlags
  • cleaningUpOldCompilationOutputs
  • linkFilelistWithDebugInfo
  • windowsRuntimeLibraryFlags

SanitizerTests.swift (6 tests)

  • sanitizerRecoverArgs
  • sanitizerArgs
  • sanitizerCoverageArgs
  • sanitizerAddressUseOdrIndicator
  • sanitizeStableAbi
  • sanitizerArgsForTargets

OptRecordTests.swift (11 tests)

  • optimizationRecordFileInSupplementaryOutputFileMap
  • optimizationRecordWithOutputFileMap
  • optimizationRecordPartialFileMapCoverage
  • optimizationRecordPathUserProvidedPath
  • optimizationRecordMultiThreadedWMOInsufficientPaths
  • optimizationRecordMultiThreadedWMOWithExplicitPaths
  • optimizationRecordMultiThreadedWMODerivedPaths
  • optimizationRecordMultiThreadedWMOWithCompleteFileMap
  • optimizationRecordSingleThreadedWMOModuleLevelEntry
  • optimizationRecordSingleThreadedWMODerivedPath
  • optimizationRecordWarningWithExplicitPathAndFileMap

TargetTests.swift (17 tests)

  • targetTriple
  • targetVariant
  • targetVariantEmitModule
  • validDeprecatedTargetiOS
  • validDeprecatedTargetWatchOS
  • clangTargetForExplicitModule
  • disableClangTargetForImplicitModule
  • environmentInferenceWarning
  • darwinToolchainArgumentValidation
  • darwinSDKVersioning
  • darwinSDKTooOld
  • darwinLinkerPlatformVersion
  • darwinSDKWithoutSDKSettings
  • darwinSDKToolchainName
  • nonDarwinSDK
  • isIosMacInterface
  • windowsOptions

ToolchainTests.swift (17 tests)

  • toolchainClangPath
  • executableFallbackPath
  • swiftHelpOverride
  • swiftClangOverride
  • swiftClangxxOverride
  • toolsDirectory
  • toolSearching
  • registrarLookup
  • findingBlockLists
  • findingBlockListVersion
  • useStaticResourceDir
  • relativeResourceDir
  • sdkDirLinuxPrioritizedOverRelativeResourceDirForLinkingSwiftRT
  • frontendTargetInfoWithWorkingDirectory
  • pluginPaths
  • workingDirectoryForImplicitOutputs
  • workingDirectoryForImplicitModules

SpecialModeTests.swift (15 tests)

  • help
  • repl
  • immediateMode
  • installAPI
  • updateCode
  • versionRequest
  • noInputs
  • printTargetInfo
  • frontendSupportedArguments
  • emitSupportedArguments
  • printOutputFileMap
  • dumpASTOverride
  • dumpASTFormat
  • deriveSwiftDocPath
  • aDDITIONAL_SWIFT_DRIVER_FLAGS

OutputJobTests.swift (24 tests)

  • recordedInputModificationDates
  • baseOutputPaths
  • outputFileMapLoading
  • findingObjectPathFromllvmBCPath
  • outputFileMapLoadingDocAndSourceinfo
  • indexUnitOutputPath
  • outputFileMapStoring
  • outputFileMapResolving
  • outputFileMapRelativePathArg
  • referenceDependencies
  • responseFileExpansion
  • responseFileExpansionRelativePathsInCWD
  • responseFileExpansionRelativePathsNotInCWD
  • responseFileTokenization
  • usingResponseFiles
  • specificJobsResponseFiles
  • emitSymbolGraphPrettyPrint
  • emitSymbolGraphSkipSynthesizedMembers
  • emitSymbolGraphSkipInheritedDocs
  • emitSymbolGraphShortenModuleNames
  • sourceInfoFileEmitOption
  • filelist
  • emitLLVMIR
  • supplementaryOutputFileMapUsage

MiscDriverTests.swift (39 tests)

  • moduleAliasingWithImplicitBuild
  • invalidModuleAliasing
  • privateInterfacePathImplicit
  • packageInterfacePathImplicit
  • profileArgValidation
  • profileSampleUseFrontendFlags
  • debugInfoForProfilingFlag
  • profileLinkerArgs
  • conditionalCompilationArgValidation
  • frameworkSearchPathArgValidation
  • multipleValidationFailures
  • dotFileEmission
  • regressions
  • diagnosticOptions
  • numThreads
  • deterministicCheck
  • warnConcurrency
  • libraryLevel
  • prebuiltModuleCacheFlags
  • relativeInputs
  • adopterConfigFile
  • extractPackageName
  • extractLibraryLevel
  • supportedFeatureJson
  • saveUnkownDriverFlags
  • emitClangHeaderPath
  • clangModuleValidateOnce
  • webAssemblyUnsupportedFeatures
  • scanDependenciesOption
  • experimentalPerformanceAnnotations
  • verifyEmittedInterfaceJob
  • verifyEmittedPackageInterface
  • loadPackageInterface
  • vfsOverlay
  • cachingBuildOptions
  • enableFeatures
  • emitAPIDescriptorEmitModule
  • emitAPIDescriptorWholeModuleOptimization
  • emitABIDescriptor

IntegrationTests.swift (kept as separate suite)

  • verboseImmediateMode (moved over as it relies on Bundle path)
  • litDriverTests
  • litDriverValidationTests
  • litInterpreterTests
  • litStdlibTests
  • litSymbolGraphFrontendTest

@cachemeifyoucan cachemeifyoucan requested a review from a team as a code owner April 13, 2026 16:49
@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch 14 times, most recently from f5a2a3f to 6225a79 Compare April 16, 2026 16:42
@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch 2 times, most recently from 72339c3 to a67a594 Compare April 27, 2026 17:56
@cachemeifyoucan cachemeifyoucan changed the title [Draft] Switching to testing with Swift Testing [Test] Switching to testing with Swift Testing Apr 27, 2026
@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test

@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch from b03358b to 5cf5543 Compare April 27, 2026 19:26
@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test

1 similar comment
@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test

@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch from ad92b5d to d05d460 Compare April 28, 2026 15:43
@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test

Add the building blocks that the Swift Testing migration depends on,
without yet migrating any test files:

- TestDriver: async-safe wrapper around Driver that dispatches blocking
  engine.build() calls off the cooperative thread pool via GCD, so
  tests run concurrently under Swift Testing without starving threads.
- TestLocalFileSystem: FileSystem wrapper with per-instance CWD, so
  concurrent tests no longer mutate the process-global working
  directory.
- DriverTestHelpers: shared test helpers (executableName, rebase,
  testInputsPath, toPath/toPathOption, envWithFakeSwiftHelp, ld stub,
  and Array<Job.ArgTemplate> extensions) that will be used after
  splitting SwiftDriverTests.swift.
- TestExpectations: expectCommandLineContains /
  expectJobInvocationMatches (successor to XCTestExtensions).
- testDiagnosticsHandler / verboseTestOutput: gated on
  SWIFT_DRIVER_TEST_VERBOSE=1 for debuggable diagnostic output.
@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch 2 times, most recently from 9620b5d to 48b25fb Compare May 4, 2026 22:15
…DriverTests.swift

Migrate all remaining XCTest-based tests in SwiftDriverTests,
SwiftOptionsTests, IncrementalImportTests, and IncrementalTestFramework
to Swift Testing, and break up the monolithic SwiftDriverTests.swift
(9,433 lines, 221 tests) into 10 thematic test files.

Assertion and suite conversions:
- XCTAssert* -> #expect / #require / Issue.record
- XCTestCase classes -> @suite structs with @test functions
- XCTSkip -> ConditionTrait (.requireMacOS, .requireDarwin, .skipWindows,
  .skipDarwin, .requireFrontendArgSupport, .requireScannerSupports*,
  .requireExplicitModuleVerifyInterface, caching / prefix-mapping traits)
- setUp/tearDown -> @suite traits or inline helpers
- Driver(args:) in tests -> TestDriver(args:) (async-safe wrapper)

SwiftDriverTests.swift split into:
- DriverOptionTests (option parsing, module naming, settings)
- CompileJobTests (compile jobs, batch mode, WMO, PCH/PCM)
- LinkJobTests (linking, dsym, LTO, linker flags)
- SanitizerTests (sanitizer arguments)
- OptRecordTests (optimization record paths)
- TargetTests (target triples, SDK versioning, platform)
- ToolchainTests (toolchain paths, resource dirs, tools)
- SpecialModeTests (REPL, immediate mode, help, version)
- OutputJobTests (output file maps, response files, symbol graphs)
- MiscDriverTests (validation, diagnostics, misc features)

Helper updates:
- XCTestExtensions.swift removed (superseded by TestExpectations.swift)
- AssertDiagnostics.swift converted to async, uses Issue.record +
  SourceLocation
- AssertDiagnosticsTests.swift converted using withKnownIssue
- MockingIncrementalCompilation.swift and IncrementalTestHarness.swift
  converted to Swift Testing
- IncrementalTestFramework files converted (SourceLocation-based)
- TestBuildConfig gains Swift Testing ConditionTraits for feature gating
- DriverExtensions.swift drops legacy test-only Driver(args:) initializer
  (callers use TestDriver)
@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-switch-to-swift-testing branch from 48b25fb to 92393ad Compare May 4, 2026 22:58
@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test

@cachemeifyoucan
Copy link
Copy Markdown
Contributor Author

@swift-ci please test windows platform

@cachemeifyoucan cachemeifyoucan merged commit bc9e869 into swiftlang:main May 5, 2026
48 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants