Skip to content

Support for re-running failed tests #22

@bflad

Description

@bflad

Certain tests may be flaky and it would be desirable to just re-run the failed tests instead of everything again. For example, this workflow is required now where the re-run command is copy-pasting elements from the results:

$ tctest pr 12530
Discovering tests for pr #12530 (https://github.com/terraform-providers/terraform-provider-aws/pull/12530)...
    TestAccAWSStorageGatewayNfsFileShare
triggering refs/pull/12530/merge for (TestAccAWSStorageGatewayNfsFileShare)...
  bflad@HOSTNAME#Aws_ProviderAwsAlternate
  build 103995 queued: https://HOSTNAME/viewQueued.html?itemId=103995

$ tctest results 103995
--- PASS: TestAccAWSStorageGatewayNfsFileShare_basic (198.28s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_ClientList (265.22s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_DefaultStorageClass (270.50s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_GuessMIMETypeEnabled (287.48s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_KMSKeyArn (303.82s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_tags (315.32s)
--- PASS: TestAccAWSStorageGatewayNfsFileShare_Squash (317.48s)
--- FAIL: TestAccAWSStorageGatewayNfsFileShare_KMSEncrypted (716.07s)
--- FAIL: TestAccAWSStorageGatewayNfsFileShare_NFSFileShareDefaults (768.03s)
--- FAIL: TestAccAWSStorageGatewayNfsFileShare_ObjectACL (775.86s)
--- FAIL: TestAccAWSStorageGatewayNfsFileShare_ReadOnly (775.84s)
--- FAIL: TestAccAWSStorageGatewayNfsFileShare_RequesterPays (791.91s)

$ tctest pr 12530 '(TestAccAWSStorageGatewayNfsFileShare_KMSEncrypted|TestAccAWSStorageGatewayNfsFileShare_NFSFileShareDefaults|TestAccAWSStorageGatewayNfsFileShare_ObjectACL|TestAccAWSStorageGatewayNfsFileShare_ReadOnly|TestAccAWSStorageGatewayNfsFileShare_RequesterPays)'
triggering refs/pull/12530/merge for (TestAccAWSStorageGatewayNfsFileShare_KMSEncrypted|TestAccAWSStorageGatewayNfsFileShare_NFSFileShareDefaults|TestAccAWSStorageGatewayNfsFileShare_ObjectACL|TestAccAWSStorageGatewayNfsFileShare_ReadOnly|TestAccAWSStorageGatewayNfsFileShare_RequesterPays)...
  bflad@HOSTNAME#Aws_ProviderAwsAlternate
  build 103998 queued: https://HOSTNAME/viewQueued.html?itemId=103998

It'd be really nice if we could so something like the following instead:

$ tctest run-failed-results 103995
# or
$ tctest pr --failed-only 12530
# or
$ tctest pr --run-failed-results=103995 12530

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions