Skip to content

[20843] Add action to generate flaky test reports - #107

Draft
EduPonz wants to merge 20 commits into
mainfrom
feature/detect_flaky_tests
Draft

[20843] Add action to generate flaky test reports#107
EduPonz wants to merge 20 commits into
mainfrom
feature/detect_flaky_tests

Conversation

@EduPonz

@EduPonz EduPonz commented Jun 12, 2024

Copy link
Copy Markdown

Description

This PR adds a flakiness_report action to create flaky tests reports in nightly workflows.
This stems from the original work done by @elianalf in:

TODO:

  • Add Windows support
  • Add macOS support

Contributor Checklist

  • Commit messages follow the company guidelines.
  • Changes are API compatible.
  • New features have been added to the versions.md and README.md files (if applicable).

Reviewer Checklist

  • The title and description correctly express the PR's purpose.
  • The Contributor checklist is correctly filled.

@EduPonz
EduPonz force-pushed the feature/detect_flaky_tests branch 6 times, most recently from 3369cbb to 976d703 Compare June 13, 2024 10:16
EduPonz added 5 commits June 14, 2024 10:53
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
@EduPonz
EduPonz force-pushed the feature/detect_flaky_tests branch from d07a07f to 1685099 Compare June 14, 2024 08:54
EduPonz added 10 commits June 17, 2024 14:45
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
@EduPonz
EduPonz force-pushed the feature/detect_flaky_tests branch from 1685099 to 35aaca1 Compare June 17, 2024 12:46
EduPonz added 4 commits June 17, 2024 15:05
…ction

Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Signed-off-by: eduponz <eduardoponz@eprosima.com>
timestamp="{$input_timestamp}">
<xsl:for-each select="//Site/Testing/Test">
<xsl:variable name="testName" select="translate(Name, '-', '_')"/>
<xsl:variable name="duration" select="Results/NamedMeasurement[@name='Execution Time']/Value"/>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is just a reminder for us: if we will have problems related to storage space of the artifacts, we may need to to create a reduced version of this .xsl file saving only the necessary information related to the test results. A possible solution is the following:

Suggested change
<xsl:variable name="duration" select="Results/NamedMeasurement[@name='Execution Time']/Value"/>
<xsl:variable name="duration" select="Results/NamedMeasurement[@name='Execution Time']/Value"/>
<xsl:variable name="status" select="@Status"/>
<xsl:variable name="output" select="Results/Measurement/Value"/>
<xsl:variable name="className" select="translate(Path, '/.', '.')"/>
<testcase classname="projectroot{$className}"
name="{$testName}"
status="{$status}"
time="{$duration}">
</testcase>
</xsl:for-each>
</testsuite>
</testsuites>
</xsl:template>
</xsl:stylesheet>

Comment thread resources/junit/junit_utils.py
Comment thread resources/junit/junit_utils.py Outdated
Comment thread resources/flaky/publisher.py Outdated
Comment thread external/upload-artifact/action.yml Outdated
Signed-off-by: eduponz <eduardoponz@eprosima.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants