-
Notifications
You must be signed in to change notification settings - Fork 36
5-Spot Scheduler - APPROVED - Software Project Contribution and Onboarding #387
Description
Please note that only FINOS members can propose new projects. If you're interested in membership, see https://www.finos.org/membership-benefits#become-a-member.
Onboarding Process
Completing an onboarding of a project into FINOS requires following these 5 main steps:
- Describing the Contribution led by contributor
- Approval led by FINOS TOC
- Preparing for Onboarding led by contributor
- Onboarding completed by FINOS Infra
- Announcement led by FINOS Marketing
1. Describing The Contribution
This is a list of questions that need to be answered by the contributor in order to allow a new project to pass to the approval stage of onboarding.
Business Problem
Modern cloud infrastructure requires dynamic scaling of compute resources based on predictable time patterns, yet most solutions are reactive rather than proactive. Organizations face several challenges:
- Cost Optimization: Expensive compute resources running 24/7 even during off-hours when not needed
- Energy Efficiency: Unnecessary power consumption from always-on infrastructure contributing to environmental impact
- Predictable Workloads: Batch processing, development environments, and scheduled tasks that follow business hours but lack intelligent scheduling
- Physical Infrastructure Management: Difficulty managing bare-metal and edge infrastructure with the same automation as cloud resources
- Compliance Requirements: Need for auditable, policy-driven resource management with strict controls over when compute resources are active
Organizations typically see 40-70% cost reduction potential for non-critical workloads, but lack tools designed specifically for time-based, proactive scheduling of physical infrastructure.
Proposed Solution
5 Spot Machine Scheduler is a Kubernetes operator (project contribution) that provides intelligent, time-based scheduling for physical machine deployments. It solves the business problem by:
- Proactive Time-Based Scheduling: Automatically provisions and de-provisions physical machines based on configurable day-of-week and hour-of-day schedules with timezone support
- Physical Infrastructure Focus: Built specifically for managing bare-metal servers, edge devices, and hybrid cloud deployments, not just cloud VMs
- Kubernetes-Native: Leverages Kubernetes operator patterns with Custom Resource Definitions (CRDs), enabling Infrastructure-as-Code and GitOps workflows
- Graceful Lifecycle Management: Includes configurable graceful shutdown periods for workload migration and emergency kill-switch functionality
- Enterprise Features: Multi-instance support for high availability, priority-based resource allocation, comprehensive observability, and k0smotron/Cluster API integration
- Cost and Energy Reduction: Eliminates 24/7 operation costs for time-sensitive workloads while ensuring resources are available exactly when needed
The project enables organizations to treat physical infrastructure with the same automation and declarative management as cloud resources, while dramatically reducing costs and environmental impact.
Why FINOS?
Alignment with FINOS Mission:
- Open Financial Services: Born from real financial services needs at RBC
- Infrastructure Modernization: Enables cost-effective cloud-native infrastructure
- Industry Collaboration: Common challenge across financial institutions
- Standards-Based: Built on CNCF/Kubernetes standards
This contribution fits into the grid computing ecosystem already identified by FINOS.
In particular, it enables the compute scheduling that complements the task scheduling
provided by iother projects, such as OpenGRIS. Discussions have already happened with
the maintainers of OpenGRIS with very positive feedback.
Value to FINOS Community:
- Solves cost optimization for market-hours workloads (trading, analytics)
- Addresses compliance requirements for resource scheduling
- Enables hybrid cloud strategies with physical infrastructure
- Promotes sustainable computing practices
Tentative Roadmap
Short-term goals (6-12 months):
- Stabilize v1.0 release with production-ready feature set
- Expand integration testing and documentation
- Build community adoption through demos, tutorials, and use case documentation
- Add enhanced observability features (centralized dashboards, Prometheus metrics expansion)
- Implement additional scheduling patterns (cron-like expressions, holiday calendars)
- Develop reference implementations for common use cases (CI/CD, batch processing, dev environments)
Medium-term goals (12-24 months):
- Multi-cluster scheduling coordination
- Advanced placement policies and resource constraints
- Integration with additional Cluster API providers beyond k0smotron
- Cost analytics and reporting features
- Plugin architecture for custom scheduling logic
- Workload-aware scheduling with Kubernetes pod integration
- Enhanced security features (mTLS, secrets management, RBAC policies)
Success metrics:
- Adoption by 10+ organizations across different industries
- Demonstrated cost savings averaging 50%+ for participating organizations
- Active contributor community with regular contributions
- Production deployments managing 1000+ scheduled machines
- Integration with major infrastructure provisioning tools
- Recognition as a standard solution for time-based infrastructure scheduling
Current State
History: 5 Spot Machine Scheduler was developed internally at RBC Capital Markets to solve the challenge of efficiently utilizing unused compute capacity during off-hours, particularly for burst scenarios like end-of-day disaster recovery capacity. The project was built by the Kubernetes Platform Engineering team and has been demonstrated internally with integration into RBC's fleet management system (Monty).
Current state:
- Version 0.1.0 with core functionality implemented
- Built using the
kube-rsframework for Kubernetes operators - Comprehensive documentation using
mkdocs - Integration with k0smotron (Mirantis Cluster API implementation)
- Features include: time-based scheduling, graceful shutdown, kill switch, priority-based processing, machine reference tracking, multi-instance support
- Demonstrated with real workloads using confidential containers (Kata) for workload isolation
- Documentation site deployed at internal RBC infrastructure
- Currently used internally with centralized control through fleet management tooling
Technology stack: Rust (kube-rs framework), Kubernetes, k0smotron/Cluster API, SSH-based machine provisioning, Prometheus metrics
Existing Materials
- GitHub Repository
- URL for the repository: To be provided upon approval - currently internal RBC GitHub Enterprise
- Project Name: 5 Spot Machine Scheduler (also referred to as "5-spot")
- @finos-admin has been given read-only permissions if private (Will be granted upon approval)
- Is Continuous Integration used? Internal CI/CD pipelines within RBC infrastructure - will migrate to GitHub Actions
- Was the project ever released? Yes
- If so, are releases public? No (currently internal)
- And what's the latest released version? v0.1.0
- Existing Project Documentation: Internal documentation site at https://rbcgithub.fg.rbc.com/pages/rbc-cm/sc90-5spot/ (MKDocs generated documentation including architecture, use cases, deployment guides)
- Does the name have a registered trademark? No
- Is there a logo? Yes
- High-Level Presentation prepared for Technical Oversight Committee: Technical demo and deep dive has been conducted internally; can be adapted for FINOS TOC (~15 mins)
- Are meetings currently held for the project? Yes - Internal development team meetings at RBC
- Are meeting minutes, agenda and attendance tracked? Informal tracking - Will establish formal processes for FINOS community
Development Team
Maintainers
| Name | Affiliation | Work Email Address | Github |
|---|---|---|---|
| Erick Bourgeois | RBC Capital Markets | erick.bourgeois@rbccm.com | @ebourgeois |
| Brad Penney | RBC Capital Markets | dan.guns@rbccm.com | @bradpenney |
| Dan Guns | RBC Capital Markets | dan.guns@rbccm.com | @dgunzy |
Confirmed contributors
| Name | Affiliation | Work Email Address | Github / GitLab username |
|---|---|---|---|
| Erick Bourgeois | RBC Capital Markets | erick.bourgeois@rbccm.com | @ebourgeois |
| Brad Penney | RBC Capital Markets | dan.guns@rbccm.com | @bradpenney |
| Dan Guns | RBC Capital Markets | dan.guns@rbccm.com | @dgunzy |
Target Contributors
We welcome contributions from a diverse range of technical professionals who work with cloud-native infrastructure, Kubernetes, and physical infrastructure management:
Primary Target Profiles:
-
Platform Engineers & Site Reliability Engineers
- Organizations managing hybrid cloud and bare-metal infrastructure
- Teams working on cost optimization and infrastructure efficiency
- Experience with Kubernetes operators, Cluster API, and infrastructure automation
- Financial services, manufacturing, education, and media production sectors with time-based workload patterns
-
Cloud Infrastructure Architects
- Professionals designing and implementing cloud-native platforms
- Teams working on sustainable computing and energy efficiency initiatives
- Organizations with predictable workload patterns (business hours, batch processing, scheduled tasks)
- Experience with Infrastructure-as-Code and GitOps practices
-
DevOps Engineers & Infrastructure Developers
- Python developers with Kubernetes operator development experience
- Teams managing CI/CD infrastructure and development environments
- Contributors interested in scheduler design, resource management, and distributed systems
- Experience with tools like kopf, Cluster API, k0smotron, or similar frameworks
-
Enterprise IT Operations
- Financial institutions managing trading infrastructure and market-hours workloads
- Educational institutions managing lab and classroom computing resources
- Organizations with compliance and governance requirements for compute resource usage
- Teams managing edge computing deployments with regional time-based patterns
-
Open Source Maintainers & Contributors
- Maintainers of related CNCF/cloud-native projects (Cluster API, scheduling tools, infrastructure operators)
- Contributors to k0smotron, Mirantis k0s, or other Kubernetes distribution projects
- Developers working on cost optimization, FinOps, or sustainability tooling
- Documentation writers and technical educators focused on infrastructure management
Organizational Profiles:
- Financial services firms with market-hours compute requirements
- Technology companies managing multi-tenant infrastructure
- Research institutions with grant-funded compute resources
- Manufacturing and IoT deployments with shift-based operations
- Cloud service providers interested in hybrid infrastructure solutions
- Enterprises transitioning from always-on to time-aware infrastructure
Contribution Areas:
- Core operator development (Python/Kubernetes)
- Integration with additional Cluster API providers and infrastructure platforms
- Enhanced scheduling algorithms and patterns
- Observability, metrics, and dashboards
- Documentation, tutorials, and use case examples
- Testing, quality assurance, and deployment automation
- Security enhancements and compliance features
- Cost analytics and reporting capabilities
We particularly encourage contributions from organizations facing similar challenges with physical infrastructure management, as real-world use cases and feedback will drive the project's evolution and ensure it meets the needs of the broader community.
Project Communication Channel(s)
- Contributor to ask maintainers which communications channels they'd like to use:
- Asynchronous
- GitHub Issues (public)
- GitHub Discussions (public)
- GitHub Team Discussions (consisting of the above described contributors)
- Public
- Private
- Mailing-list (groups.io)
- FINOS Slack Channel (consisting of the above described contributors)
- General (public) (supply channel name)
- Leadership (private) (supply channel name)
- Synchronous
- Recurring meetings
Understanding FINOS Onboarding Requirements
As a project onboarding into FINOS, you will need to familiarize yourself and your contributor team with the following materials:
- FINOS overview (if necessary)
- FINOS Maintainers cheatsheet
- FINOS Project/Standards Governance
- FINOS Project Lifecycle
Record The Contribution (FINOS Infra)
- (optional) Identify and Assign FINOS Strategic Advisor
- Submit contribution to LF by opening a ticket via https://jira.linuxfoundation.org/browse/SS and marking contribution as "Exploratory"; attach a summary of the Business Problem and Proposed Solution (above) of the project.
2. Approval
The FINOS Technical Oversight Committee (TOC) is responsible for approving FINOS project contributions; feel free to check their contribution principles.
If needed, the TOC will request a follow up either via GitHub Issue comments or by inviting project leads to one of their recurrent meetings.
Tasks (for FINOS Infra/TOC)
- Prioritise this issue on the TOC Backlog
- Assign this issue to @eddie-knight
- Add the
ready-for-tsclabel - TOC to invite contributors to present their project
- FINOS TOC approves/rejects the contribution
- Ask @jgavronsky to mark contribution as "Engaged" within LF systems
- (optional) If additional socialization is required, the Executive Director may bring projects to the FINOS Governing Board
- Update the contribution status to "Engaged" by sending another email to LF Legal Representative with the name of the project and its new status.
TOC Findings / Report
- The proposal itself was broadly seen as valuable and aligned with TOC goals.
- Fits with ongoing work around governance, standardization, etc.
- Discussion was about clarifying scope, tightening definitions, and ensuring it doesn't overlap with existing initiatives; with emphasize on avoiding duplication and aligning with the rest of HPC efforts.
3. Preparing For Onboarding
Before the FINOS team can onboard your project, there are a few housekeeping that need to be taken care of. These must be completed by the contributor, with help if required from the FINOS Infra.
Kick-off meeting
- Set up kick-off meeting with project leads
- Run kick-off meeting
- Walk through the checklist in part 1, ensure all the questions are answered and remove items that don't apply
- Write and send contribution proposal announcement (optional - see below)
Proposal (Lead Maintainer)
-
Lead maintainer to send out announcement to community@finos.org using this template:
Dear FINOS Community, We would like to propose a new FINOS project. Please review the proposal details at (_TODO: add link to the GitHub issue proposal_). If you're interested in participating, please :+1: the GitHub issue proposal and drop a comment with your name, org and email Thanks a lot,
Logo / Trademarks
- Sign the project contribution agreement to allow FINOS to act on behalf of the contributor for accounts related to the project (e.g., GitHub, domain names, social media) and to optionally manage trademark assignment
- Request logo design from
help@finos.org(if needed) - The codebase doesn’t include any patent or copyright that conflicts with FINOS Governance and bylaws. (Infra team to validate with FINOS Legal team if anything important is raised)
FINOS Project Blueprint
- finos-admin is Admin of the GitHub repository to transfer
- Apply project blueprint contents - see our docs on collaboration infrastructure
- README.md file contains the sections in the README Template
- Project FINOS badge in README (incubating)
- OpenSSF Passing Badge
- License in README
- Contributing in README
-
CONTRIBUTING.md -
LICENSE(replace{}placeholders) - Ensure that the proper project governance is in the CONTRIBUTING.md file
- All incubating criteria are checked and documented below
Add documentation here
4. FINOS Onboarding
This is performed by FINOS Infra once the three previous stages are complete, with support from the contributor and the FINOS Infra team.
Maintainers, Contributors and CLAs
- Identify other existing contributors (assuming there's a contribution history (eg Git history)
- Check if maintainers and other contributors are all covered by FINOS CLA
- Engage with FINOS Legal team to figure out what’s needed to cover all maintainers and contributors with FINOS CLA
- Reach out to contributors and employers to coordinate CLA signatures
- Invite GitHub usernames to GitHub FINOS Org
- Create
<project-name>-maintainersGitHub team and invite users
Validation (only if code is contributed)
- Backup (even with screenshot) GitHub permissions of the repository to transfer
- Check GitHub repository transfer requirements:
- finos-admin has
Adminto all repositories to transfer - finos-admin ia allowed to transfer repositories out of the org
- if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
- finos-admin has
- The codebase doesn’t have HIGH or CRITICAL CVEs across direct and transitive libraries
- The codebase doesn’t have any unfriendly licenses across direct and transitive libraries
- (optional - if a build system is provided) The build process runs successfully
- The codebase doesn’t include any patent or copyright that conflicts with FINOS Governance and bylaws (to be validated with FINOS Legal team)
- All incubating criteria are met (review documentation provided above)
Code transfer
- Transfer all code assets as GitHub repositories under github.com/finos
- Rename main branch to
main(instead ofmaster) - Configure
finos-admins(Maintainrole) andfinos-staff(Triagerole) team permissions
Project Communication Channel(s)
- Create the identified communication channels during infra set up
- Create mailing-list on lists.finos.org (if requested)
- Enable Hubspot Sync for all project mailing lists created
- Aggregate mailing lists to community@finos.org
- Update marketing lists (see
Email ListandEmailfilter fields), particularly Hubspot all community list
- Create Slack channels if requested.
- Link communication channels linked front-and-center in the project README.md
Repository setup
- Enable EasyCLA
- Add project to metadata
- Add identities, orgs and affiliations to metadata (deprecated by EasyCLA)
- Add logo to FINOS landscape
- Create
stagingbranch onfinos/finos-landscape - Merge
finos/metadatachanges on master (will udpdatelandscape.ymlinfinos/finos-landscape) - Create PR from
stagingbranch onfinos/finos-landscape - Review Netlify preview
- Merge PR
- Create
- Add project maintainers emails to finos-project-maintainers@finos.org list
- Add project maintainers GitHub usernames to the project-maintainers Team
- Ask @jgavronsky to mark contribution as "Active" within LF systems
- (best effort) Update release coordinates and code namespace to include
finos - Check that CVE (and preferably static code analysis, if applicable) scanning is in place
- Enable automated dependency update, using Renovate
- Enable branch protection (
Require a pull request before merging) - (optional) Enable meeting attendance tracking
- (optional) Onboard into legend.finos.org/studio
5. Announcement
(Lead: Project Lead and FINOS Infra team)
- Update the contribution status to "Active" by sending another email to LF Legal Representative with the name of the project and its new status.
- Lead maintainer works with FINOS marketing to send out announcement to announce@finos.org , checkout announcement template at the Contribution page
- Notify FINOS marketing (@grizzwolf + finos-marketing internal Slack channel)
Metadata
Metadata
Assignees
Type
Projects
Status