Skip to content

Fix failing checks: e2e tests, welcome action#383

Merged
becky-gilbert merged 9 commits intomainfrom
fix-e2e-tests
Mar 25, 2026
Merged

Fix failing checks: e2e tests, welcome action#383
becky-gilbert merged 9 commits intomainfrom
fix-e2e-tests

Conversation

@becky-gilbert
Copy link
Copy Markdown
Contributor

@becky-gilbert becky-gilbert commented Mar 17, 2026

This PR fixes some bugs in the e2e tests and results page:

e2e/experiment.test.js

  • Incorrect URL was used in the API mock (should be **/api/${expInfo.shortName}/getPercentileRank instead of **/api/results).
  • Removed test for "Loading..." text - this state is too brief to test reliably.
  • Mocked API response was returning the results in an incorrect format - needs to be wrapped in a resData object.
  • API mock needs to be set up before the page click, so that it catches the request (no reload needed).

e2e/results.test.js

  • In the beforeEach, use page.waitForResponse to make sure that the API post request has finished before checking results. Previously this was just using the DOM content to check that the results are ready, but the POST request and database update could still have been pending, which was causing intermittent errors.

templates/experiments/[EXP]/src/web page/src/results.js

  • Fix logic error that determines whether the page should show the "Oops! Something went wrong" message. The previous logic used a falsy check for data.percentileRank and data.summary_stat, so when those values were 0 (valid), it produced the error message. Now it checks for null/undefined. (I didn't change the falsy !data.totalRows check. A value of 0 is valid, but that case is handled earlier - it produces a "Results aren't available" message). This issue was causing one of the e2e tests to fail.

This also fixes some bugs in the failing 'welcome' Github Workflow: .github/workflows/welcome.yml

  • The actions/first-interaction action updated its API to use underscores, but the config was still using the old hyphenated names.
  • Removes the unnecessary actions/checkout@v3 step. The actions/first-interaction only needs the GitHub token to post a comment, not the repository contents, and the repo cloning step takes a very long time.
  • Adds a manual dispatch option to the welcome action, for easier testing (this action only runs on newly-opened PRs/issues, which makes it hard to test changes).

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 17, 2026

🦋 Changeset detected

Latest commit: 15ad62a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@pushkin-templates/exp-basic Patch
@pushkin-templates/exp-grammaticality-judgment Patch
@pushkin-templates/exp-lexical-decision Patch
@pushkin-templates/exp-self-paced-reading Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@becky-gilbert becky-gilbert changed the title Fix e2e tests Fix failing checks: e2e tests, welcome action Mar 18, 2026
add changesets for templates file changes
@becky-gilbert
Copy link
Copy Markdown
Contributor Author

@cherriechang do you mind reviewing this? All of the fixes were in test files or GH action config, so it shouldn't conflict with anything that you're working on. The only exception is this change to the templates/experiments/[exp]/src/web page/src/results.js files: 58c32b7 which was causing incorrect results page content when the percentile rank or summary stat is a valid 0 value. We could revert that commit here if you've already addressed that issue somewhere else.

Copy link
Copy Markdown
Contributor

@cherriechang cherriechang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thank you!! Everything makes sense and was easy to follow from the PR conversation.

@becky-gilbert becky-gilbert merged commit 41f4142 into main Mar 25, 2026
5 checks passed
@github-actions github-actions bot mentioned this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working exp templates Relates to experiments templates testing/CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants