feat(hpa): Add horizontal pod autoscaling for DragonFly instances#343
feat(hpa): Add horizontal pod autoscaling for DragonFly instances#343smunukutla-mycarrier wants to merge 15 commits intodragonflydb:mainfrom
Conversation
… test-single in makefile
Signed-off-by: Siva Munukutla <smunukutla@mycarrier.io>
|
Hi @smunukutla-mycarrier, thanks for the PR! can you please resolve the conflicts? |
Signed-off-by: Siva Munukutla <smunukutla@mycarrier.io>
|
@Abhra303 done. Thanks! |
|
@Abhra303 I would really appreciate it if we could merge this soon. We're looking forward to rolling out autoscaling for dragonfly. Please let me know if there any issues/concerns. Thanks! :) |
|
@smunukutla-mycarrier, when do you think we can merge this PR? Where I am working, this is a feature we want to implement. |
|
@ldiego73 I'm not a maintainer on this repo. I'm also waiting for a review from the maintainers, when they get a chance. |
|
bumping this, please merge. |
|
Adding my support for this feature. Please merge. |
|
Hi @smunukutla-mycarrier, the test is not fixed yet. Also can you elaborate more about the reason for supporting HPA? Dragonfly doesn't support multiple masters. So this can only scale reads. Is this the reason you want to support hpa for? |
|
@Abhra303 thanks for the feedback. Yes, we need to be able to (horizontally) scale secondary nodes automatically based on traffic/utilization. I'll push a fix for the tests this weekend, hopefully. |
Signed-off-by: Siva Munukutla <smunukutla@mycarrier.io>
|
@Abhra303 I also merged latest from main branch and resolved conflicts. Please run the workflow again when you have some time. |
|
When is this being merged? It will be a great addition. Autoscaling is a great way to have peace of mind. I don't want to be worried about needing someone scaling othe pods manually when traffic increases. |
api/v1alpha1/dragonfly_types.go
Outdated
| // +kubebuilder:validation:Maximum=100 | ||
| TargetCPUUtilizationPercentage *int32 `json:"targetCPUUtilizationPercentage,omitempty"` | ||
|
|
||
| // Target memory utilization percentage |
There was a problem hiding this comment.
does this makes sense ?
how would adding replica reduce memory usage ? as data is replicated to all.
There was a problem hiding this comment.
100% agreed. It doesn't make sense to provide memory utilization as a target. I've updated it along with the tests.
| // +kubebuilder:validation:Optional | ||
| // +kubebuilder:validation:Minimum=1 | ||
| // +kubebuilder:validation:Maximum=100 | ||
| TargetCPUUtilizationPercentage *int32 `json:"targetCPUUtilizationPercentage,omitempty"` |
There was a problem hiding this comment.
Note that adding replicas can only increase your read throughput
There was a problem hiding this comment.
Yes. The feature to scale horizontally is to accommodate cache reads as traffic increases - reducing load on master and existing replicas. It would be great to handle that automatically rather than updating replicas manually.
There was a problem hiding this comment.
@ashotland please review and run the workflow when you get a chance. The tests passed for me locally.
…c and related files
There was a problem hiding this comment.
Pull request overview
This pull request adds Horizontal Pod Autoscaling (HPA) support to the Dragonfly operator, enabling automatic scaling of Dragonfly instances based on CPU/memory utilization or custom metrics. The implementation includes API changes to support autoscaler configuration, controller logic to manage HPA resources, proper replica count handling during autoscaling transitions, and comprehensive e2e tests.
Key Changes:
- Added
AutoscalerSpecto the Dragonfly CRD with support for min/max replicas, metrics, and scaling behavior - Implemented HPA resource generation and lifecycle management in the controller
- Enhanced StatefulSet replica handling to preserve HPA-managed replica counts
- Added RBAC permissions for managing HPA resources
Reviewed changes
Copilot reviewed 13 out of 16 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| api/v1alpha1/dragonfly_types.go | Added AutoscalerSpec type with validation, preservation annotations for Kubernetes objects |
| api/v1alpha1/zz_generated.deepcopy.go | Generated deepcopy methods for AutoscalerSpec |
| internal/resources/resources.go | Implemented HPA generation logic and StatefulSet replica initialization based on autoscaler config |
| internal/controller/dragonfly_instance.go | Added HPA deletion logic and StatefulSet replica preservation during reconciliation |
| internal/controller/dragonfly_controller.go | Added HPA ownership tracking and requeue logic for HPA deletion |
| manifests/dragonfly-operator.yaml | Updated CRD with autoscaler schema (removed descriptions to reduce size) and RBAC rules |
| config/rbac/role.yaml | Added HPA resource permissions |
| config/samples/v1alpha1_dragonfly_autoscaler.yaml | Added sample autoscaler configuration |
| e2e/*.go | Added ImagePullPolicy and improved secret cleanup in tests |
| README.md | Updated to advertise HPA support |
| Makefile | Added test-single target and maxDescLen=0 to reduce CRD size |
| cmd/main.go | Minor log formatting fix |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Abhradeep Chakraborty <abhradeep@dragonflydb.io>
|
@smunukutla-mycarrier I see some fields in the manifest files are missing. Can you please check? |
This pull request resolves #320. It adds autoscaling support to Dragonfly operator, allowing Horizontal Pod Autoscaler to be used with Dragonfly instances.
Autoscaling (HPA) support
AutoscalerSpectoDragonflySpecindragonfly_types.go, allowing users to configure HPA settings such as enabling autoscaling, min/max replicas, target CPU/memory utilization, scaling behavior, and metrics.dragonfly_instance.goto create, update, or delete HPA resources based on theAutoscalerSpec, including cleanup of HPA when autoscaling is disabled and preservation of replica counts during transitions.role.yamland controller ownership ofHorizontalPodAutoscalerobjects.API and code generation
dragonfly_types.goandzz_generated.deepcopy.goto support the new autoscaler configuration.Documentation and examples
README.mdto advertise HPA support as a main feature.v1alpha1_dragonfly_autoscaler.yamldemonstrating how to configure autoscaling for a Dragonfly instance.E2E tests and minor improvements
ImagePullPolicyto several test cases for consistency.cmd/main.go.List of E2E tests for autoscaler