NETOBSERV-2387: Backend tests migration#2586
NETOBSERV-2387: Backend tests migration#2586oliver-smakal wants to merge 16 commits intonetobserv:mainfrom
Conversation
|
Skipping CI for Draft Pull Request. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2586 +/- ##
==========================================
- Coverage 72.50% 72.30% -0.21%
==========================================
Files 107 107
Lines 11482 11482
==========================================
- Hits 8325 8302 -23
- Misses 2658 2676 +18
- Partials 499 504 +5
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Important Review skippedToo many files! This PR contains 266 files, which is 116 over the limit of 150. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (34)
📒 Files selected for processing (266)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@oliver-smakal: This pull request references NETOBSERV-2387 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
77f49b5 to
e9dff92
Compare
e9dff92 to
77f49b5
Compare
77f49b5 to
299451f
Compare
There was a problem hiding this comment.
thanks @oliver-smakal , some questions/comments.
/cc @jotak @OlivierCazade @leandroberetta @jpinsonneau
JSYK, in terms of code the files in testdata and all the code has been already reviewed in openshift-tests-private except for couple of new files (version_checker.go, backend_suite_test.go) , besides those new files, the most important thing here to review here go.mod for new dependencies.
Description
This PR is intended to show how the e2e backend migration of tests could look like and where we can discuss more details.
Changes compare to the openshift-tests-private repo:
ginkgodirectly without wrapperBeforeSuitepossible, but we need to be carefull about not usingcompat_otp.NewCLIand similar utilities which haveg.BeforeEachcallsTestcases were migrated, there were some changes made due to the golanci linter complaining. Minor things were applied, however there were some things such as passing heavy structs without using * which I don't really see as a problem -> I have adjusted the Makefile to omit the integration-tests from linting/testing.
This PR can be tested with flexy cluster via the Network Observability Backend Tests.
Review tips
Important files to definitely check:
To check changes in the testcases checkout the latest
openshift-tests-privateon the main branch and use:git diff --no-index ~/Repos/openshift-tests-private/test/extended/netobserv/test_flowcollector.go test_flowcollector.goto check file diff or
to check whole dirs.
How to run locally
Either:
or if the ginkgo cli is installed:
Other standard flags of ginkgo such as
--dry-runor-valso work.Example output
The following is an example output of of a run:
This implementation cannot control the
[1776103444] Backend Suite - 3/7488 specsline. I don't think it will be issue in any way, as the rest of the report can make it really clear to us what is actually being run. For running it in prow it should also not be a problem if we preserve the way it works with theopenshift-test-privaterepo and use junit report.More examples:
How it could be used in CI.
Currently the
openshift-test-privateimplementation uses junit to handle the result.See lines 398-459 that junit result is generated and parsed and later is used by the openshift-e2e-test-qe-report step to fail the prow job if necessary). For the nice output in prow, it seems like the function
handle_resultin theopenshift-extended-teststep is used as it seems to be formatting and renaming the junit file with the help ofhandleresult.pypython script.Though this is probably not directly reusable for implementation as the junit in
openshift-test-privatedoes not use default ginkgo implementation, we could use the same idea and use the junit output with minor transformation in prow to report result in a nice way.Alternative approaches
We could create something a bit more custom using more low level ginkgo functionality like in commit fbb6fc36b17bc37b6eb847b8e51f42d26a9b29d8, to remove things like the
[1776103444] Backend Suite - 3/7488 specsline. However, I don't think it would be worth the tradeoff of not using the default ginkgo runner viaRunSpecs()as we would use the possibility to run tests in parallel and some other features.