Hyperfoil bulk by ncau · Pull Request #1890 · kruize/autotune

ncau · 2026-04-28T19:57:26Z

Description

Adding a basic test for the bulk api using hyperfoil. Creates a single bulk request with the following JSON:

          {
            "datasource": "prometheus-1",
            "metadata_profile": "cluster-metadata-local-monitoring",
            "measurement_duration": "24h"
          }

It then requests the Job ID to look for a successful run looking at the status and the total experiments. Currently the users are set to 1 (a single request with these parameters. We can alter this however we would like to include more requests.

Fixes # (issue)

Type of change

Bug fix
[ x] New feature
Docs update
Breaking change (What changes might users need to make in their application due to this PR?)
Requires DB changes

How has this been tested?

Please describe the tests that were run to verify your changes and steps to reproduce. Please specify any test configuration required.

[ x] New Test X
Functional testsuite

Test Configuration
ITCP Openshift cluster

Checklist 🎯

Additional information

Include any additional information such as links, test results, screenshots here

Summary by Sourcery

Add a Hyperfoil-based performance test for the bulk API and update the Jetty dependency version.

New Features:

Introduce a Hyperfoil scenario to exercise the bulk API, including job creation and status retrieval.

Build:

Update Jetty dependency version in the Maven build configuration.

Tests:

Add an automated Hyperfoil load test configuration for the bulk API lifecycle (submit bulk job and query its status).

Bumps org.eclipse.jetty:jetty-server from 12.0.12 to 12.0.32. --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-server dependency-version: 12.0.32 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>

…se.jetty-jetty-server-12.0.32 Bump org.eclipse.jetty:jetty-server from 12.0.12 to 12.0.32

sourcery-ai · 2026-04-28T19:57:34Z

Reviewer's Guide

Adds a new Hyperfoil-based bulk API test scenario and updates the Jetty dependency version.

Sequence diagram for Hyperfoil-driven bulk API test execution

sequenceDiagram
    actor Tester
    participant Hyperfoil
    participant OpenshiftRoute as Openshift_route
    participant KruizeService as Kruize_service
    participant Jetty as Jetty_12_0_32
    participant BulkAPI as Bulk_API_handler

    Tester->>Hyperfoil: Start_kruize-bulk-hyperfoil_yaml_test
    Hyperfoil->>OpenshiftRoute: HTTP_bulk_request_batch
    OpenshiftRoute->>KruizeService: Forward_bulk_request
    KruizeService->>Jetty: Dispatch_HTTP_request
    Jetty->>BulkAPI: Invoke_bulk_API_handler
    BulkAPI-->>Jetty: Bulk_response_payload
    Jetty-->>KruizeService: HTTP_response
    KruizeService-->>OpenshiftRoute: Forward_response
    OpenshiftRoute-->>Hyperfoil: HTTP_response
    Hyperfoil-->>Tester: Report_latency_throughput_metrics

File-Level Changes

Change	Details	Files
Introduce a Hyperfoil scenario to exercise the /bulk API end‑to‑end.	Define a Hyperfoil workload named kruize-bulk-api-test targeting the cluster’s kruize API route. Configure a constantRate phase that posts to /bulk with a minimal JSON payload to create a bulk job. Extract the returned job_id from the bulk creation response and store it in a variable. Insert a thinkTime delay before status verification to allow the job to be created. Issue a GET /bulk?job_id=${job.id} request and extract summary.status and summary.total_experiments from the response body.	`tests/scripts/hyperfoil_tests/kruize-bulk-hyperfoil.yaml`
Update Jetty server dependency to a newer patch version.	Bump the jetty-version Maven property from 12.0.12 to 12.0.32 to align builds with the newer Jetty release.	`pom.xml`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey - I've left some high level feedback:

The Hyperfoil test hardcodes the OpenShift route hostname, which will make this difficult to run in other environments; consider parameterizing the host (e.g., via a property or environment variable) instead of embedding a cluster-specific URL.
In the Hyperfoil scenario you capture job.status and total.experiments but never assert on them; adding explicit checks (e.g., expecting a certain status or minimum experiments) would make the test detect regressions instead of just collecting values.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The Hyperfoil test hardcodes the OpenShift route hostname, which will make this difficult to run in other environments; consider parameterizing the host (e.g., via a property or environment variable) instead of embedding a cluster-specific URL.
- In the Hyperfoil scenario you capture `job.status` and `total.experiments` but never assert on them; adding explicit checks (e.g., expecting a certain status or minimum experiments) would make the test detect regressions instead of just collecting values.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

Copilot

Pull request overview

Adds an initial Hyperfoil benchmark to exercise the Bulk API endpoint and updates the Jetty dependency version in the Maven build.

Changes:

Added a Hyperfoil YAML scenario to POST /bulk and then GET /bulk status.
Bumped jetty-version from 12.0.12 to 12.0.32 in pom.xml.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
tests/scripts/hyperfoil_tests/kruize-bulk-hyperfoil.yaml	Introduces a Hyperfoil benchmark intended to validate basic `/bulk` job creation + status retrieval.
pom.xml	Updates Jetty version property used by Jetty dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kusumachalasani · 2026-04-30T09:02:21Z

@ncau Can you please include more description about the PR.

chandrams · 2026-05-05T10:14:42Z

@ncau Can you resolve the conflicts and add description and documentation for this test.

ncau · 2026-05-05T21:27:27Z

@kusumachalasani added more info let me know if I need to clarify anything else and resolved the merge conflict.

ncau · 2026-05-06T20:22:58Z

@chandrams @kusumachalasani added new documentation should help clear up issues.

kusumachalasani · 2026-05-15T06:23:50Z

@ncau Please automate the whole process of running the test and print the results by the end of the script. I would suggest use jbang-hyperfoil or install hyperfoil itself through script.

ncau · 2026-05-18T10:43:05Z

@kusumachalasani review the install script please

chandrams · 2026-05-19T07:12:59Z

@@ -0,0 +1,93 @@
+# Kruize Bulk API — Hyperfoil Load Test


@ncau Please move this into tests/scripts/local_monitoring_tests/bulk_load_test. Also the test should be completely automated from doing all the setup required to running the tests and processing the results.

I updated the location and had it run the script automatically. What do we need from a results point of view, in terms of processing? is there a specific format we need the output in?

@kusumachalasani I 'm not familiar with hyperfoil output here, can you suggest on what needs to be checked in terms of metrics captured to automate the validation of this test.

@ncau This test should be completely automated to run on Jenkins, so all setup, test run and validation to check if results are as expected needs to be added. On success we exit with 0 and on failures we exit with 1 in other tests for perf lab jenkins

dependabot Bot and others added 3 commits March 5, 2026 22:26

Merge pull request kruize#1839 from kruize/dependabot/maven/org.eclip…

6e5d5ac

…se.jetty-jetty-server-12.0.32 Bump org.eclipse.jetty:jetty-server from 12.0.12 to 12.0.32

adding basic hyperfoil bulk test

19f54c5

ncau requested review from chandrams and Copilot April 28, 2026 19:57

Copilot started reviewing on behalf of ncau April 28, 2026 19:57 View session

sourcery-ai Bot reviewed Apr 28, 2026

View reviewed changes

Copilot AI reviewed Apr 28, 2026

View reviewed changes

chandrams requested a review from kusumachalasani April 30, 2026 08:56

chandrams added this to Monitoring May 5, 2026

chandrams modified the milestones: Kruize 0.12.0 Release, Kruize 0.11.0 Release May 5, 2026

chandrams assigned ncau May 5, 2026

Merge branch 'mvp_demo' into hyperfoilBulk

aad6c6a

add placeholder url and documentation for the hyperofoil test

14afdb8

rbadagandi1 moved this to In Progress in Monitoring May 11, 2026

rbadagandi1 moved this from In Progress to Under Review in Monitoring May 12, 2026

rbadagandi1 added the remote_monitoring label May 12, 2026

add a script to install hyperfoil

2f8cbbb

chandrams reviewed May 19, 2026

View reviewed changes

move files and automate full test run

911eee0

Conversation

ncau commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How has this been tested?

Checklist 🎯

Additional information

Summary by Sourcery

Uh oh!

sourcery-ai Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for Hyperfoil-driven bulk API test execution

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kusumachalasani commented Apr 30, 2026

Uh oh!

chandrams commented May 5, 2026

Uh oh!

ncau commented May 5, 2026

Uh oh!

ncau commented May 6, 2026

Uh oh!

kusumachalasani commented May 15, 2026

Uh oh!

ncau commented May 18, 2026

Uh oh!

chandrams May 19, 2026

Choose a reason for hiding this comment

Uh oh!

ncau May 20, 2026

Choose a reason for hiding this comment

Uh oh!

chandrams May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ncau commented Apr 28, 2026 •

edited

Loading

sourcery-ai Bot commented Apr 28, 2026 •

edited

Loading