test(integration): purge orphan datasets in annotation cleanup fixtures

[henrycgbaker]

Goal

Tests that exercise full teardown (empty dataset_id) can accumulate orphan datasets across runs (if dataset uses non-canonical name and teardown does not specify. it, or if exits midway). Argilla refuses to delete a workspace while any dataset is linked, so the next teardown_resources call hits 409 ConflictError and the suite fails.

This PR targets orphaned datasets in test fixtures, but not production.

Scope

Test fixtures only - no production code changes. Production callers own their dataset_id and don't accumulate orphans, so teardown_resources keeps its current targeted behaviour.

Implementation

New helper tests/integration/_argilla_cleanup.purge_workspace_datasets(client, ws_base) deletes every dataset in a workspace.

clean_slate (test_annotation_setup.py) and clean_environment (test_annotation_import.py) call the purge for each configured workspace before teardown_resources. This keeps the fix at the right layer - test-induced state pollution is handled by test fixtures, not by widening the production teardown contract.

Earlier draft of this PR pushed the orphan-purge into core/annotation/setup.py so a full teardown would delete every dataset in each workspace before deleting the workspace. Rejected because:

• Multi-tenant blast radius: in shared Argilla deployments, that wipes datasets owned by other runs/users
• Silent contract change: same call signature, different (more destructive) behaviour
• Wrong layer: the symptom is test-fixture state, not a production-API gap

If a production-side purge is wanted later, the right shape is an explicit purge_workspace(ws_base, *, confirm=True) function, not an implicit branch in teardown_resources.

Testing

• pytest tests/integration/test_annotation_setup.py tests/integration/test_annotation_import.py -m "integration and annotation" -> 14 passed (with the test user pre-purged; the user-leak failure is the separate test isolation bug fixed in test(integration): function-scoped fixtures for order-independent annotation tests #166).
• Lint + mypy green.

References

• Stacked on test(integration): slim dev-stack integration tests #164. Surfaced by the preflight added in test(integration): progressive preflight for annotation stack #163.
• Test isolation refactor (function-scoped fixtures, user-purge) follows in test(integration): function-scoped fixtures for order-independent annotation tests #166.

[henrycgbaker]

open question (@saschagobel let me know if you have thoughts otherwise I'll return to this this evening):

better in tests just to remove ALL datasets cleanly (global, non-scoped)

for ds in client.datasets:
    ds.delete()

rather than scope to AnnotationSettings.workspace_dataset_map ?

NB pending this decision -> update #166 workflow accordingly