diff --git a/.gitbook.yml b/.gitbook.yml deleted file mode 100644 index 2ea2632..0000000 --- a/.gitbook.yml +++ /dev/null @@ -1,4 +0,0 @@ -root: ./docs/ - -​structure: - readme: README.md \ No newline at end of file diff --git a/README.md b/README.md index ac3ce34..ada5cf1 100644 --- a/README.md +++ b/README.md @@ -64,6 +64,10 @@ Then open in VS Code → **Reopen in Container** (or `Cmd+Shift+P` → "Dev Cont The devcontainer pulls the pre-built image from GHCR — no local Docker build required. +#### Cursor with GitHub Codespaces + +Use **Cursor** with a **Codespace** via **`gh codespace ssh`** and **Remote - SSH**. See **[Cursor + Codespaces + devcontainer](docs/cursor-remote-devcontainer.md)**. + ### Option 3: Docker run (headless) ```bash diff --git a/docs/.gitbook/assets/1.png b/docs/.gitbook/assets/1.png deleted file mode 100644 index e32acd9..0000000 Binary files a/docs/.gitbook/assets/1.png and /dev/null differ diff --git a/docs/.gitbook/assets/10 (1).png b/docs/.gitbook/assets/10 (1).png deleted file mode 100644 index b873690..0000000 Binary files a/docs/.gitbook/assets/10 (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/10.png b/docs/.gitbook/assets/10.png deleted file mode 100644 index b873690..0000000 Binary files a/docs/.gitbook/assets/10.png and /dev/null differ diff --git a/docs/.gitbook/assets/2.png b/docs/.gitbook/assets/2.png deleted file mode 100644 index c6807db..0000000 Binary files a/docs/.gitbook/assets/2.png and /dev/null differ diff --git a/docs/.gitbook/assets/3.png b/docs/.gitbook/assets/3.png deleted file mode 100644 index ba29e19..0000000 Binary files a/docs/.gitbook/assets/3.png and /dev/null differ diff --git a/docs/.gitbook/assets/4.png b/docs/.gitbook/assets/4.png deleted file mode 100644 index fed2269..0000000 Binary files a/docs/.gitbook/assets/4.png and /dev/null differ diff --git a/docs/.gitbook/assets/5.png b/docs/.gitbook/assets/5.png deleted file mode 100644 index 6947345..0000000 Binary files a/docs/.gitbook/assets/5.png and /dev/null differ diff --git a/docs/.gitbook/assets/6.png b/docs/.gitbook/assets/6.png deleted file mode 100644 index 847d6e0..0000000 Binary files a/docs/.gitbook/assets/6.png and /dev/null differ diff --git a/docs/.gitbook/assets/8.png b/docs/.gitbook/assets/8.png deleted file mode 100644 index 05e32e1..0000000 Binary files a/docs/.gitbook/assets/8.png and /dev/null differ diff --git a/docs/.gitbook/assets/9.png b/docs/.gitbook/assets/9.png deleted file mode 100644 index e47e4c3..0000000 Binary files a/docs/.gitbook/assets/9.png and /dev/null differ diff --git a/docs/.gitbook/assets/RW_Authors_Asset.png b/docs/.gitbook/assets/RW_Authors_Asset.png deleted file mode 100644 index 8d3302d..0000000 Binary files a/docs/.gitbook/assets/RW_Authors_Asset.png and /dev/null differ diff --git a/docs/.gitbook/assets/Screenshot from 2024-01-15 07-53-41.png b/docs/.gitbook/assets/Screenshot from 2024-01-15 07-53-41.png deleted file mode 100644 index ccb257a..0000000 Binary files a/docs/.gitbook/assets/Screenshot from 2024-01-15 07-53-41.png and /dev/null differ diff --git a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-09-19.png b/docs/.gitbook/assets/Screenshot from 2024-01-15 10-09-19.png deleted file mode 100644 index 181385f..0000000 Binary files a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-09-19.png and /dev/null differ diff --git a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-12-00.png b/docs/.gitbook/assets/Screenshot from 2024-01-15 10-12-00.png deleted file mode 100644 index bd78734..0000000 Binary files a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-12-00.png and /dev/null differ diff --git a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-14-04.png b/docs/.gitbook/assets/Screenshot from 2024-01-15 10-14-04.png deleted file mode 100644 index 92a605b..0000000 Binary files a/docs/.gitbook/assets/Screenshot from 2024-01-15 10-14-04.png and /dev/null differ diff --git a/docs/.gitbook/assets/codecollection-development-Development Flow.drawio.png b/docs/.gitbook/assets/codecollection-development-Development Flow.drawio.png deleted file mode 100644 index ec794d4..0000000 Binary files a/docs/.gitbook/assets/codecollection-development-Development Flow.drawio.png and /dev/null differ diff --git a/docs/.gitbook/assets/codecollection-development-image-layersdrawio.svg b/docs/.gitbook/assets/codecollection-development-image-layersdrawio.svg deleted file mode 100644 index 653b2b5..0000000 --- a/docs/.gitbook/assets/codecollection-development-image-layersdrawio.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
RunWhen Robot Runtime Base Image
Base image that is used when building images for the SaaS platform
RunWhen Robot Runtime Base Image...
CodeCollection Devtools 
Adds common development tools, filesystem structures, sudo etc.
CodeCollection Devtools...
CodeCollection Development Image
Allows users to add their own software to the image when necessary
CodeCollection Development Image...
Base Image
Base Image
Intermediate Image
Intermediate Image
Final DevContainer Image
Final DevContainer Image \ No newline at end of file diff --git a/docs/.gitbook/assets/codecollection-development.drawio (1).svg b/docs/.gitbook/assets/codecollection-development.drawio (1).svg deleted file mode 100644 index ef801e7..0000000 --- a/docs/.gitbook/assets/codecollection-development.drawio (1).svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
SLI
(sli.robot)
SLI...
TaskSet
(runbook.robot)
TaskSet...
Engineering Assistants
(Eager Edgar)
Engineering Assistan...
Workflows / Webhooks /
Alerts
Workflows / Webhooks...
Users
Users
Time Interval 
(eg. 30s)
Time Interval...
RunWhen Platform Metrics Engine
RunWhen Platform Met...
Issues & Next Steps
Issues & Next Steps
Report Data
Report Data \ No newline at end of file diff --git a/docs/.gitbook/assets/codecollection-development.drawio.svg b/docs/.gitbook/assets/codecollection-development.drawio.svg deleted file mode 100644 index abf2bb7..0000000 --- a/docs/.gitbook/assets/codecollection-development.drawio.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
SLI
(sli.robot)
SLI...
TaskSet
(runbook.robot)
TaskSet...
Engineering Assistants
(Eager Edgar)
Engineering Assistan...
Workflows / Webhooks /
Alerts
Workflows / Webhooks...
Users
Users
Time Interval 
(eg. 30s)
Time Interval...
RunWhen Platform Metrics Engine
RunWhen Platform Met...
Issues & Next Steps
Issues & Next Steps
Report Data
Report Data \ No newline at end of file diff --git a/docs/.gitbook/assets/codecollection-development.png b/docs/.gitbook/assets/codecollection-development.png deleted file mode 100644 index 784e8e9..0000000 Binary files a/docs/.gitbook/assets/codecollection-development.png and /dev/null differ diff --git a/docs/.gitbook/assets/image (1) (1) (1) (1).png b/docs/.gitbook/assets/image (1) (1) (1) (1).png deleted file mode 100644 index ab5dece..0000000 Binary files a/docs/.gitbook/assets/image (1) (1) (1) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (1) (1) (1).png b/docs/.gitbook/assets/image (1) (1) (1).png deleted file mode 100644 index a9b9602..0000000 Binary files a/docs/.gitbook/assets/image (1) (1) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (1) (1).png b/docs/.gitbook/assets/image (1) (1).png deleted file mode 100644 index adb4f62..0000000 Binary files a/docs/.gitbook/assets/image (1) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (1).png b/docs/.gitbook/assets/image (1).png deleted file mode 100644 index 0ad9e14..0000000 Binary files a/docs/.gitbook/assets/image (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (10).png b/docs/.gitbook/assets/image (10).png deleted file mode 100644 index f2ee17f..0000000 Binary files a/docs/.gitbook/assets/image (10).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (11).png b/docs/.gitbook/assets/image (11).png deleted file mode 100644 index b8f7580..0000000 Binary files a/docs/.gitbook/assets/image (11).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (12).png b/docs/.gitbook/assets/image (12).png deleted file mode 100644 index 18e0e62..0000000 Binary files a/docs/.gitbook/assets/image (12).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (13).png b/docs/.gitbook/assets/image (13).png deleted file mode 100644 index 1318b49..0000000 Binary files a/docs/.gitbook/assets/image (13).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (14).png b/docs/.gitbook/assets/image (14).png deleted file mode 100644 index 4d4333a..0000000 Binary files a/docs/.gitbook/assets/image (14).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (15).png b/docs/.gitbook/assets/image (15).png deleted file mode 100644 index 8c82148..0000000 Binary files a/docs/.gitbook/assets/image (15).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (16).png b/docs/.gitbook/assets/image (16).png deleted file mode 100644 index 89b3c3d..0000000 Binary files a/docs/.gitbook/assets/image (16).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (17).png b/docs/.gitbook/assets/image (17).png deleted file mode 100644 index ef89c5e..0000000 Binary files a/docs/.gitbook/assets/image (17).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (18).png b/docs/.gitbook/assets/image (18).png deleted file mode 100644 index cca13d2..0000000 Binary files a/docs/.gitbook/assets/image (18).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (19).png b/docs/.gitbook/assets/image (19).png deleted file mode 100644 index 2acf97f..0000000 Binary files a/docs/.gitbook/assets/image (19).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (2) (1) (1).png b/docs/.gitbook/assets/image (2) (1) (1).png deleted file mode 100644 index f3555c2..0000000 Binary files a/docs/.gitbook/assets/image (2) (1) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (2) (1).png b/docs/.gitbook/assets/image (2) (1).png deleted file mode 100644 index bbc356a..0000000 Binary files a/docs/.gitbook/assets/image (2) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (2).png b/docs/.gitbook/assets/image (2).png deleted file mode 100644 index 5ec123a..0000000 Binary files a/docs/.gitbook/assets/image (2).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (20).png b/docs/.gitbook/assets/image (20).png deleted file mode 100644 index f26acd9..0000000 Binary files a/docs/.gitbook/assets/image (20).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (21).png b/docs/.gitbook/assets/image (21).png deleted file mode 100644 index 5a6bb4c..0000000 Binary files a/docs/.gitbook/assets/image (21).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (22).png b/docs/.gitbook/assets/image (22).png deleted file mode 100644 index 5a6bb4c..0000000 Binary files a/docs/.gitbook/assets/image (22).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (3) (1).png b/docs/.gitbook/assets/image (3) (1).png deleted file mode 100644 index 753ccb4..0000000 Binary files a/docs/.gitbook/assets/image (3) (1).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (3).png b/docs/.gitbook/assets/image (3).png deleted file mode 100644 index c4fbeaa..0000000 Binary files a/docs/.gitbook/assets/image (3).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (4).png b/docs/.gitbook/assets/image (4).png deleted file mode 100644 index 6afa1a9..0000000 Binary files a/docs/.gitbook/assets/image (4).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (5).png b/docs/.gitbook/assets/image (5).png deleted file mode 100644 index b073fda..0000000 Binary files a/docs/.gitbook/assets/image (5).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (6).png b/docs/.gitbook/assets/image (6).png deleted file mode 100644 index 7a49a51..0000000 Binary files a/docs/.gitbook/assets/image (6).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (7).png b/docs/.gitbook/assets/image (7).png deleted file mode 100644 index d0f7135..0000000 Binary files a/docs/.gitbook/assets/image (7).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (8).png b/docs/.gitbook/assets/image (8).png deleted file mode 100644 index 2c6ccdd..0000000 Binary files a/docs/.gitbook/assets/image (8).png and /dev/null differ diff --git a/docs/.gitbook/assets/image (9).png b/docs/.gitbook/assets/image (9).png deleted file mode 100644 index 46e2c81..0000000 Binary files a/docs/.gitbook/assets/image (9).png and /dev/null differ diff --git a/docs/.gitbook/assets/image.png b/docs/.gitbook/assets/image.png deleted file mode 100644 index 2a4efd0..0000000 Binary files a/docs/.gitbook/assets/image.png and /dev/null differ diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 1617d8e..0000000 --- a/docs/README.md +++ /dev/null @@ -1,31 +0,0 @@ -# Introduction - -
- -At the heart of our platform and [FOSS](https://en.wikipedia.org/wiki/Free\_and\_open-source\_software) offerings is our community of authors. They play a critical role in curating open source collections of troubleshooting commands that the community can either use directly, through [RunWhen Local](https://docs.runwhen.com/public/runwhen-local/introduction-runwhen-local), or on our [platform](https://docs.runwhen.com/public/runwhen-platform/getting-started-with-runwhen-platform) with Engineering Assistants. Our goal is to build the world’s largest open collection of operational code with a level of testing, documentation and metadata that will surpass the libraries at hyperscale companies. It's incredibly challenging - if not impossible - to replicate these multi-thousand person efforts within most companies… but an open source community can. - -If any of this sounds interesting to you we'd be excited to have you in the community. Below you'll find a glossary of terms and resources to get started as an author, as well as some details on our rewards program. - -If you have any questions feel free to reach out to our team on [Slack](https://runwhen.slack.com/join/shared\_invite/zt-1l7t3tdzl-IzB8gXDsWtHkT8C5nufm2A#/shared-invite/email). - -

Author CodeCollection Flow Overview

- -### CodeCollections - -CodeCollections are version-controlled repositories that follow a consistent file structure and house collections of codebundles which are the core building blocks of functionality across our ecosystem. CodeCollections can be indexed, searched and leveraged by both the [RunWhen Platform](https://docs.runwhen.com/public/runwhen-platform/getting-started-with-runwhen-platform) and [RunWhen Local](https://docs.runwhen.com/public/runwhen-local/introduction-runwhen-local), serving as the primary functionality for both. - -### CodeBundles - -CodeBundles are combinations of source code, metadata, and dynamic configuration that can be used to perform various actions. They most commonly consist of Python, Robot Framework and bash. Codebundles are the basis of SLIs, SLOs and TaskSets on the [platform](https://docs.runwhen.com/public/runwhen-platform/getting-started-with-runwhen-platform), and for wrapping commands with metadata for use in [RunWhen Local](https://docs.runwhen.com/public/runwhen-local/introduction-runwhen-local). - -### Troubleshooting Commands - -At the root of many CodeBundles, are troubleshooting commands wrapped by metadata and configuration. If you've got some troubleshooting commands you keep in your clipboard or personal wiki that you'd like to share, or would like to browse troubleshooting commands for various cloud native products like Vault, Cert Manager, Redis, and Kubernetes then head over to our [discussions](https://github.com/orgs/runwhen-contrib/discussions). Troubleshooting commands can be freely shared and discussed with the community and leveraged by our FOSS tool [RunWhen Local](https://docs.runwhen.com/public/runwhen-local/introduction-runwhen-local) for you and your whole team. - -### Contributing - -We'd love to collaborate and have you be a part of our community. To get started head over to our [contributing](authors-program-details/contributing.md) doc which also has info for author rewards. - -### Communication - -For communicating with us, feel free to join and reach out to us on our public [slack](https://runwhen.slack.com/), [discord](https://discord.gg/Ut7Ws4rm8Q) or join the [discussions](https://github.com/orgs/runwhen-contrib/discussions) here! diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md deleted file mode 100644 index ff377c1..0000000 --- a/docs/SUMMARY.md +++ /dev/null @@ -1,41 +0,0 @@ -# Table of contents - -* [Introduction](README.md) - -## CodeCollection Development - -* [Requirements](codecollection-development/prerequisites.md) -* [Getting Started](codecollection-development/running-your-first-codebundle.md) -* [CodeBundle Basics](codecollection-development/running-your-first-codebundle-locally.md) -* [CodeBundle Development Checklist](codecollection-development/codebundle-development-checklist.md) -* [CodeBundle Examples](codecollection-development/codebundle-examples/README.md) - * [Health Check a URL](codecollection-development/codebundle-examples/health-check-a-url.md) - * [Scan For Crashing Pods](codecollection-development/codebundle-examples/scan-for-crashing-pods.md) -* [Generation Rules](codecollection-development/generation-rules/README.md) - * [Kubernetes Generation Rule Examples](codecollection-development/generation-rules/kubernetes-generation-rule-examples.md) - * [Azure Generation Rule Examples](codecollection-development/generation-rules/azure-generation-rule-examples.md) - * [AWS Generation Rule Examples](codecollection-development/generation-rules/aws-generation-rule-examples.md) - * [GCP Generation Rule Examples](codecollection-development/generation-rules/gcp-generation-rule-examples.md) -* [Sandbox / Lab Resources](codecollection-development/sandbox-resources/README.md) - * [JenkinsX](codecollection-development/sandbox-resources/jenkinsx.md) - * [Postgres (Operator and test database)](codecollection-development/sandbox-resources/postgres-operator-and-test-database.md) -* [CodeBundle Testing](codecollection-development/codebundle-testing.md) -* [CodeBundle Debugging](codecollection-development/codebundle-debugging.md) - -*** - -* [RunWhen Keywords](runwhen-keywords.md) - -## Authors Program Details - -* [Author Rewards](authors-program-details/author-rewards.md) -* [Code of Conduct](authors-program-details/code-of-conduct.md) -* [Contributing](authors-program-details/contributing.md) - -## Additional Resources - -* [CodeBundle Status Project Board](https://github.com/orgs/runwhen-contrib/projects/1) - -*** - -* [CodeBundle Request Form](https://github.com/runwhen-contrib/codecollection-registry/issues) diff --git a/docs/authors-program-details/author-rewards.md b/docs/authors-program-details/author-rewards.md deleted file mode 100644 index 857fae9..0000000 --- a/docs/authors-program-details/author-rewards.md +++ /dev/null @@ -1,23 +0,0 @@ -# Author Rewards - -Below you'll find details on the various rewards associated with being a CodeCollection author in our community! - -### Bounties - -While the royalty payments are the primary focus of the Code Collection Authors program, from time to time the RunWhen team may offer one-time "bounty" bonuses in return for specific code bundles built to our specifications and on our client’s timeframes, e.g. "can anyone build an SLI and TaskSet for AWS SQS next week?" - -To take a bounty, go have a look through our [issues](https://github.com/runwhen-contrib/rw-cli-codecollection/issues) marked with the `has bounty` label and assign yourself to the issue (provided it has no other assignees). Bounty size is generally related to the priority labels, indicating that a client may need it asap. - -### Royalties - -Authors in the program are eligible for royalties from the work they contribute. For more details on the royalties model please reach out to us directly. - -### SWAG - -Customized hoodies, hats, carabiners, dry erase markers, water bottles, USB drives, AND MORE! Regular contributors receive customized swag so they can proudly promote their contributions to RunWhen! - - - - - -For further details check this google [doc](https://docs.google.com/document/d/1oB1gEKvKhWQSyJ6AypeYOUpdqBTlGAE1Dflu7AECtyE/edit#heading=h.8w0xz5rsgbjo). diff --git a/docs/authors-program-details/code-of-conduct.md b/docs/authors-program-details/code-of-conduct.md deleted file mode 100644 index 9a9edaf..0000000 --- a/docs/authors-program-details/code-of-conduct.md +++ /dev/null @@ -1,5 +0,0 @@ -# Code Of Conduct - -Program CodeCollections and RunWhen CodeCollections all follow the [Stack Overflow Code Of Conduct](https://stackoverflow.com/conduct). - -Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the lead of the author program [jonathan.funk@runwhen.com](mailto:jonathan.funk@runwhen.com). diff --git a/docs/authors-program-details/contributing.md b/docs/authors-program-details/contributing.md deleted file mode 100644 index 38d4bde..0000000 --- a/docs/authors-program-details/contributing.md +++ /dev/null @@ -1,30 +0,0 @@ -# Contributing - -There are many different ways to contribute to our community which are outlined below. You may also choose to have your work be private to you/your team. Here is an overview of the steps you'll be taking as you work through our author documentation: - -1. Set up a local development environment and get a 'hello world' CodeCollection up and running on your machine. At this point, you can begin exploring the various Robot/Python libraries we've built or start building your own. [See these docs](https://docs.runwhen.com/public/v/runwhen-authors/codecollection-development/getting-started/running-your-first-codebundle) -2. Work with a member of the RunWhen team (over slack) to get it published on the platform so it is built in to images and indexed by RunWhen's search. At this point, your SLIs and TaskSets can be used by anyone on the platform and your open source contributions can be showcased on the platform. -3. When ready, work with a member of the RunWhen team about moving your repo to the runwhen-contrib github organization, setting up a regression test environment (provided by RunWhen) and a support agreement. At this point, you start participating in the revenue share / royalties program. - -More on the 2023 CodeCollection authors program is here: - -
Resources
Technical documentation on starting your own CodeCollectionThese pages
Commercial documentation on the royalty / revenue share programsThe google doc here
Slack workspace to collaborate with RunWhen engineers and other CodeCollection Authors

Slack invite link here.

Feel free to introduce yourself on #general, and please join #codecollection-discussion and #codebundle-updates channels.

Community Discord Server where you can chat with the community and get help.Discord invite link here.
Talk with our founderBook a time to talk here
- -## Development Process Overview - -If you'd like to jump in see [these docs](https://docs.runwhen.com/public/v/runwhen-authors/codecollection-development/getting-started/running-your-first-codebundle), but here's an overview of what development looks like - -1. Set up your CodeCollection repository from the template provided to get started quickly. -2. Set up a local development environment using the RunWhen DevTools container and link it to the CodeCollection repository to get all of the same tools we use in development. -3. Iterate through local development, running your SLI or TaskSet on your local machine until the first CodeBundle is in reasonable shape. Commit it to your CodeCollection repo and create a tagged v0.0.1 release. -4. Work with a member of the RunWhen team to get your CodeCollection repository registered with the platform. This includes it in a platform CI/CD pipeline and makes it available for users to configure in their SLXs. -5. Build a test Workspace that shows off the initial capabilities of your CodeCollection. The RunWhen team can help make this available to all platform users, and help promote it. -6. Iterate for your next release. - -## DevTools Container - -Our DevTools Container is intended for use with VSCode running locally in a [VS Code / VS Code Server local container topology](https://code.visualstudio.com/docs/devcontainers/containers). While we encourage this approach, you can always run the image directly, described at the bottom of [this page](https://docs.runwhen.com/public/v/runwhen-authors/codecollection-development/getting-started/running-your-first-codebundle). - -## Coding Standards - -While coding standards for Python are well documented, coding standards for Robot Framework are not as well understood. Docs for those are coming shortly. diff --git a/docs/codecollection-development/codebundle-debugging.md b/docs/codecollection-development/codebundle-debugging.md deleted file mode 100644 index 623af74..0000000 --- a/docs/codecollection-development/codebundle-debugging.md +++ /dev/null @@ -1,18 +0,0 @@ -# CodeBundle Debugging - -## CodeBundle Image Builds - -When testing your CodeBundles in the RunWhen Platform, new commits made to yourCodeCollection repository trigger image builds within the Platform. These images can often take 10-20 minutes to complete, and during this time, SLXs using these CodeBundles will be running on an older version of your code. - -To assist with visibility and debugging of CodeBundle builds, Authors can be added to the **CodeCollection Authors** workspace in the RunWhen platform. This workspace provides tools for Authors to use when determining if an image is still building, if it has errored out, and so on. If you would like to be added to this workspace, or have a request for more troubleshooting tasks, please reach out through [Slack](https://runwhen.slack.com/join/shared_invite/zt-1l7t3tdzl-IzB8gXDsWtHkT8C5nufm2A#/shared-invite/email). - -

CodeCollection Authors Workspace

- -### Using the VS Code Plugin - -Authors can also interact with the **CodeCollection Authors** workspace through the [RunWhen VS Code Plugin](https://docs.runwhen.com/public/runwhen-platform/feature-overview/vs-code-plugin). The workspace will show up in the selector as **CodeCollection Authors** or **runwhen-authors.** - -

Listing available Tasks with the VS Code Plugin

- -

Running a Task with the VS Code Plugin

- diff --git a/docs/codecollection-development/codebundle-development-checklist.md b/docs/codecollection-development/codebundle-development-checklist.md deleted file mode 100644 index 6a6d61a..0000000 --- a/docs/codecollection-development/codebundle-development-checklist.md +++ /dev/null @@ -1,205 +0,0 @@ -# CodeBundle Development Checklist - -The items in this list should be reviewed when releasing a CodeBundle for general use. These are less about CodeBundle functionality (which should be thoroughly tested), and more about basic linting checks. This list will eventually be converted into an automated tool. - -## Basic Checks - -* **Settings** - * Documentation - A basic description of the CodeBundle functionality - * Metadata - * Author - A GitHub username of the author - * Display Name - The "pretty name" of the CodeBundle - * Supports - All of the platforms/technologies/languages that this CodeBundle Supports - * Suite Setup - The keyword that is called before any Task runs - -``` -*** Settings *** -Documentation Triages issues related to a deployment and its replicas. -Metadata Author stewartshea -Metadata Display Name Kubernetes Deployment Triage -Metadata Supports Kubernetes,AKS,EKS,GKE,OpenShift - -Suite Setup Suite Initialization - -``` - -Much of this data helps display details about the CodeBundle within the platform and in the [Registry](https://registry.runwhen.com) - -{% embed url="https://registry.runwhen.com/CodeCollection/rw-cli-codecollection/k8s-deployment-healthcheck/tasks/" %} - -* **Suite Setup** - * Import User Variable - Import all non-senseitive configuration values - * Import Secret - Import sensitive data / configuration values - * Set Suite Variable - Set all imported or required user variables for all Tasks - -``` -*** Keywords *** -Suite Initialization - ${kubeconfig}= RW.Core.Import Secret - ... kubeconfig - ... type=string - ... description=The kubernetes kubeconfig yaml containing connection configuration used to connect to cluster(s). - ... pattern=\w* - ... example=For examples, start here https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/ - ${DEPLOYMENT_NAME}= RW.Core.Import User Variable DEPLOYMENT_NAME - ... type=string - ... description=Used to target the resource for queries and filtering events. - ... pattern=\w* - ... example=artifactory - ${NAMESPACE}= RW.Core.Import User Variable NAMESPACE - ... type=string - ... description=The name of the Kubernetes namespace to scope actions and searching to. - ... pattern=\w* - ... example=my-namespace - ${CONTEXT}= RW.Core.Import User Variable CONTEXT - ... type=string - ... description=Which Kubernetes context to operate within. - ... pattern=\w* - ... example=my-main-cluster - Set Suite Variable ${kubeconfig} ${kubeconfig} - Set Suite Variable ${KUBERNETES_DISTRIBUTION_BINARY} ${KUBERNETES_DISTRIBUTION_BINARY} - Set Suite Variable ${CONTEXT} ${CONTEXT} - Set Suite Variable ${NAMESPACE} ${NAMESPACE} - Set Suite Variable ${DEPLOYMENT_NAME} ${DEPLOYMENT_NAME} -``` - -* **Tasks** - * Documentation - describes the specific task function - * Tags - additional tags that help with indexing of the tasks - * `RW.Core.Add Pre To Report` - For TaskSets / runbook.robot files, content must be added to the report with this keyword. - * `RW.Core.Push Metric` - For Health / sli.robot files, a metric is pushed back to the RunWhen platform with this keyword - -``` -*** Tasks *** -Check Deployment Log For Issues with `${DEPLOYMENT_NAME}` - [Documentation] Fetches recent logs for the given deployment in the namespace and checks the logs output for issues. - [Tags] - ... fetch - ... log - ... pod - ... container - ... errors - ... inspect - ... trace - ... info - ... deployment - ... ${DEPLOYMENT_NAME} - ${logs}= RW.CLI.Run Bash File - ... bash_file=deployment_logs.sh - ... cmd_override=./deployment_logs.sh | tee "${SCRIPT_TMP_DIR}/log_analysis" - ... env=${env} - ... secret_file__kubeconfig=${kubeconfig} - ... timeout_seconds=180 - ... include_in_history=false - ${history}= RW.CLI.Pop Shell History - RW.Core.Add Pre To Report - ... Recent logs from Deployment ${DEPLOYMENT_NAME} in Namespace ${NAMESPACE}:\n\n${logs.stdout} - RW.Core.Add Pre To Report Commands Used: ${history} -``` - -## Advanced Checks - -### Issues & Next Steps - -The power of RunWhen CodeBundles comes in with **Issues** and **Next Steps.** A Task can raise an Issue that provides specific detail to either a RunWhen Engineering Assistant, or for a service owner to review for further action. - -* Issues - * Severity - 1 = Critical, 2 = Error / Major, 3 = Warning/Minor, 4 = Informational - * Expected - The expected state from the Task - * Actual - The actual state from the Task - * Title - The title of the issue, with emphasis on details about _What_ and _Where_ - * Reproduce Hint - A string, or reference to the command, that was used to determine this result. - * Details - Specific details about the issue. Usually a subset of the data that is posted in the report. - * Next Steps - A `\n` deliminted list of suggested next steps. These should often look like another Task that may exist in a workspace. If someone came to you with this issue, what would you suggest they do next? - -Issues can be simple, advanced, or complex. - -#### Simple Issues - -A simple issue may provide very direct and static content, for example: - -``` - RW.Core.Add Issue title=Errors in Flux Controller Reconciliation - ... severity=3 - ... expected=No errors in Flux controller reconciliation. - ... actual=Flux controllers have errors in their reconciliation process. - ... reproduce_hint=Run flux_reconcile_report.sh manually to see the errors. - ... next_steps=Inspect Flux logs to determine which objects are failing to reconcile. - ... details=${process.stdout} -``` - -{% embed url="https://registry.runwhen.com/CodeCollection/rw-cli-codecollection/k8s-fluxcd-reconcile/tasks/" %} - -{% embed url="https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-fluxcd-reconcile/runbook.robot#L25-L33" %} - -#### Advanced Issues - -This simple example is easy to extend in another example where the severity and title may change: - -``` - IF $deployment_status["available_condition"]["status"] == "False" or $deployment_status["ready_replicas"] == "0" - ${item_next_steps}= RW.CLI.Run Bash File - ... bash_file=workload_next_steps.sh - ... cmd_override=./workload_next_steps.sh "${deployment_status["available_condition"]["message"]}" "Deployment" "${DEPLOYMENT_NAME}" - ... env=${env} - ... include_in_history=False - RW.Core.Add Issue - ... severity=1 - ... expected=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` should have minimum availability / pod. - ... actual=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` does not have minimum availability / pods. - ... title= Deployment `${DEPLOYMENT_NAME}` in Namespace `${NAMESPACE}` is unavailable - ... reproduce_hint=View Commands Used in Report Output - ... details=Deployment `${DEPLOYMENT_NAME}` has ${deployment_status["ready_replicas"]} ready pods and needs ${deployment_status["desired_replicas"]}:\n`${deployment_status}` - ... next_steps=${item_next_steps.stdout} - ELSE IF $deployment_status["unavailable_replicas"] > 0 and $deployment_status["available_condition"]["status"] == "True" and $deployment_status["progressing_condition"]["status"] == "False" - RW.Core.Add Issue - ... severity=3 - ... expected=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` should have ${deployment_status["desired_replicas"]} pods. - ... actual=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` has ${deployment_status["ready_replicas"]} pods. - ... title= Deployment `${DEPLOYMENT_NAME}` in Namespace `${NAMESPACE}` has less than the desired availability - ... reproduce_hint=View Commands Used in Report Output - ... details=Deployment `${DEPLOYMENT_NAME}` has minimum availability, but has unready pods:\n`${deployment_status}` - ... next_steps=Inspect Deployment Warning Events for `${DEPLOYMENT_NAME}` - END - IF $deployment_status["desired_replicas"] == 1 - RW.Core.Add Issue - ... severity=4 - ... expected=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` should have more than 1 desired pod. - ... actual=Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` is configured to have only 1 pod. - ... title= Deployment `${DEPLOYMENT_NAME}` in Namespace `${NAMESPACE}` is not configured to be highly available. - ... reproduce_hint=View Commands Used in Report Output - ... details=Deployment `${DEPLOYMENT_NAME}` is only configured to have a single pod:\n`${deployment_status}` - ... next_steps=Get Deployment Workload Details For `${DEPLOYMENT_NAME}` and Add to Report\nAdjust Deployment `${DEPLOYMENT_NAME}` spec.replicas to be greater than 1. - END -``` - -{% embed url="https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-deployment-healthcheck/runbook.robot#L248-L282" %} - -#### Complex Issues - -A much more advanced approach is to provide dynamic issue details depending on the output of the script or code that identified the problem, for example: - -``` - - ${recommendations}= RW.CLI.Run Cli - ... cmd=echo '${container_restart_analysis.stdout}' | awk '/Recommended Next Steps:/ {flag=1; next} flag' - ... env=${env} - ... include_in_history=false - IF $recommendations.stdout != "" - ${recommendation_list}= Evaluate json.loads(r'''${recommendations.stdout}''') json - IF len(@{recommendation_list}) > 0 - FOR ${item} IN @{recommendation_list} - RW.Core.Add Issue - ... severity=${item["severity"]} - ... expected=Containers should not be restarting for Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` - ... actual=We found containers with restarts for Deployment `${DEPLOYMENT_NAME}` in namespace `${NAMESPACE}` - ... title=${item["title"]} - ... reproduce_hint=${container_restart_analysis.cmd} - ... details=${item["details"]} - ... next_steps=${item["next_steps"]} - END - END - END -``` - -{% embed url="https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-deployment-healthcheck/runbook.robot#L84-L97" %} diff --git a/docs/codecollection-development/codebundle-examples/README.md b/docs/codecollection-development/codebundle-examples/README.md deleted file mode 100644 index 72d9a94..0000000 --- a/docs/codecollection-development/codebundle-examples/README.md +++ /dev/null @@ -1,2 +0,0 @@ -# CodeBundle Examples - diff --git a/docs/codecollection-development/codebundle-examples/health-check-a-url.md b/docs/codecollection-development/codebundle-examples/health-check-a-url.md deleted file mode 100644 index 9678021..0000000 --- a/docs/codecollection-development/codebundle-examples/health-check-a-url.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: >- - This page assumes you've created a repo from the template: - https://github.com/runwhen-contrib/codecollection-template ---- - -# Health Check a URL - -This approach to checking if a URL is available and timley is demonstrated using a Python & Robot Framework oriented design. In this example we will create a starter codebundle that checks if a URL is available (returning HTTP 200) and has a response latency below a configurable threshold. - -* First we write our python function which will be imported as a Robot Keyword; we'll place it in a python file called `checkers.py` - -

File Structure

- -* Here's the source code: - -

Python Keyword Source to check a URL

- -* And the Robot file itself: - -

HTTP Code Bundle

- -We can test this using the `ro` utility tool in the devtools container by running: - -`ro runbook.robot` - -Output: - -

HTTP Check Output

- -And we're done! From here you could look at examining the URL response more closely, provide more detailed output with differing report sections and formats for the platform and users. - -Source: [https://github.com/runwhen-contrib/codecollection-template/tree/demo/codebundles/url-healthcheck](https://github.com/runwhen-contrib/codecollection-template/tree/demo/codebundles/url-healthcheck) diff --git a/docs/codecollection-development/codebundle-examples/scan-for-crashing-pods.md b/docs/codecollection-development/codebundle-examples/scan-for-crashing-pods.md deleted file mode 100644 index aa2e256..0000000 --- a/docs/codecollection-development/codebundle-examples/scan-for-crashing-pods.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: >- - This page assumes you've created a repo from the template: - https://github.com/runwhen-contrib/codecollection-template ---- - -# Scan For Crashing Pods - -In this example we'll look at taking a more "ops" approach and implement a pod healthcheck in Bash, and then wrap it in the Robot Keyword to demonstrate how you can wrap your pre-existing Bash scripts. - -* First We'll write the bash that fetches the pods from a given namespace, and tells us if any are crashlooping: - -

Pod debug script

- -Then we simply call this bash script from Robot using ther `Run Process` Keyword: - -
- - - -And the output we get when running it using `ro` : - -

Pod Health Check Output

- -Now you know how to take all your bash files you've had hidden away and run them in Robot for use in the codecollection - sharing is caring! - -Source: [https://github.com/runwhen-contrib/codecollection-template/tree/demo/codebundles/pod-healthcheck](https://github.com/runwhen-contrib/codecollection-template/tree/demo/codebundles/pod-healthcheck) diff --git a/docs/codecollection-development/codebundle-testing.md b/docs/codecollection-development/codebundle-testing.md deleted file mode 100644 index 3f65326..0000000 --- a/docs/codecollection-development/codebundle-testing.md +++ /dev/null @@ -1,194 +0,0 @@ -# CodeBundle Testing - -## Environment Testing Flow - -The common testing flow is to test locally (from your own developer environment, interacting with your test infrastructure). Once complete, if your CodeBundle contains [generation-rules](generation-rules/ "mention"), those should be tested against your test infrastructure. If that successfully generates the necsesary RunWhen Platform configurations, they can be uploaded to a RunWhen Platform Workspace for testing from the SaaS service. - -

High Level CodeBundle Development Testing Flow

- -{% hint style="info" %} -Note: The screenshots and code used in this example has been provided from an Azure specific CodeBundle that uses the `az` cli for client authentication. References to secrets and specific configuration data will vary depending on the CodeBundle purpose. -{% endhint %} - - - -## Testing Tools - -### RO (Robot CLI) - -The `ro` cli utility is a very simple wrapper for the running robot files with a standardized output folder, which ensures that output files are: - -* not committed to source code -* easily accessible from your development environment and the local server on port 3000 - -#### **Running sli/runbook code with `ro`** - -The `ro` cli can be involked from within the CodeBundle folder: - -``` -~/rw-cli-codecollection/codebundles/azure-aks-triage$ ro sli.robot -============================================================================== -Sli :: Checks AKS Configuration and returns a 1 when healthy, or 0 when not... -============================================================================== -Check for Resource Health Issues Affecting AKS Cluster `aks-cl-1` ... . -..... ------------------------------------------------------------------------------- -Sli :: Generates a composite score about the health of an AKS clus... | PASS | -4 tasks, 4 passed, 0 failed -============================================================================== -Output: /workspace/robot_logs/sli-output.xml -Log: /workspace/robot_logs/sli-log.html -Report: /workspace/robot_logs/sli-report.html -``` - -#### **Accessing detailed trace logs** - -When running sli.robot and runbook.robot files, the detailed trace log output is a crucial tool in the development workflow, assisible through http://0.0.0.0:3000 (or whichever port CodeSpaces or equivalent development environment exposes). - -

Reviewing Robot Trace Logs

- -### Task (taskfile.dev) - -Most RunWhen Authors have started to leverage Taskfiles for managing testing tasks. CodeBundles should be self-contained in their own folders, and ideally with a `.test` subfolder that contains helpful Taskfiles. These are intended to: - -* speed up testing within your own infrastructure -* assist others in reproducing the test scenario (if trying to validate or improve the CodeBundles) - - Each Taskfile may differ from one another depending on the infrastructure components that the CodeBundle supports and is tested against. For example, one such Taskfile might have the following tasks related to setting up and tearing down Terraform infrastructure to build an AKS cluster: - -``` -~/rw-cli-codecollection/codebundles/azure-aks-triage/.test$ task -l -task: Available tasks for this project: -* build-infra: Build test infrastructure -* build-terraform-infra: Run terraform apply -* check-and-cleanup-terraform: Check and clean up deployed Terraform infrastructure if it exists -* check-terraform-infra: Check if Terraform has any deployed infrastructure in the terraform subdirectory -* check-unpushed-commits: Check if outstanding commits or file updates need to be pushed before testing. -* clean: Run cleanup tasks -* clean-rwl-discovery: Check and clean up RunWhen Local discovery output -* cleanup-terraform-infra: Cleanup deployed Terraform infrastructure -* default: Run/refresh config -* delete-slxs: Delete SLX objects from the appropriate URL -* generate-rwl-config: Generate RunWhen Local configuration (workspaceInfo.yaml) -* run-rwl-discovery: Run RunWhen Local Discovery on test infrastructure -* upload-slxs: Upload SLX files to the appropriate URL -* validate-generation-rules: Validate YAML files in .runwhen/generation-rules -``` - -As you can see, there are a number of helpful tasks here, some of which might be easily portable across multiple CodeBundles (such as `upload-slxs` or `delete-slxs`), some which require minor adjustment from one CodeBundle to the next (such as `generate-rwl-config` ) and others which might not apply at all (such as `build-terraform-infra` or `cleanup-terraform-infra` if the infrastructure uses a different provisioning mechanism). - -For this reason, existing Taskfiles should be reviewed and a README.MD should be updated so that other users can leverage them effectively. - -## Local Testing - -### CodeBundle Testing - -Once the appropriate test infrastructure has been deployed, most local testing is performed by setting the necessary environment variables and authentication steps, followed by using the `ro` utility to run the `sli.robot` or `runbook.robot` files. All required environment variables and authentication prerequisites should be documentd in the CodeBundle `README.md`. - -Once the code is functioning as expected, [generation-rules](generation-rules/ "mention")can be created (if appropriate). - -### Generation Rule Testing - -RunWhen Local can be easily used to test your CodeBundle Generation Rules if it contains the following: - -* a `.runwhen` folder with at least one Generation Rule and corresponding templates -* a .test/Taskfile with `generate-rwl-config` and `run-rwl-discovery` - -These tasks will generate a RunWhen Local configuration file that uses your CodeBundle and your test infrastructure to generate configuration. This will create a `workspaceInfo.yaml` file as well as an `output` directory with the generated configuration files. These generated files are in the `.gitignore` and will not be uploaded to your CodeCollection. - -To run the task: - -``` -~/rw-cli-codecollection/codebundles/azure-aks-triage/.test $ task -Checking for uncommitted changes in and .runwhen, excluding '.test'... -√ -No uncommitted changes in specified directories. ------------- -Checking for unpushed commits in and .runwhen, excluding '.test'... -√ -No unpushed commits in specified directories. ------------- -Stopping and removing existing container RunWhenLocal... -Cleaning up output directory... -Starting new container RunWhenLocal... -b4c2039c8b6690b8d596b1871b72395392d6bb2400e5e458e050a1cf22ae4c3a -Running workspace builder script in container... -Workspace builder REST service isn't available yet; waiting and trying again. -Workspace builder version: 0.7.4 -... -Discovery started at: 2024-11-04 22:06:37 -Discovering resources... -Workspace builder completed successfully. -Starting cheat sheet rendering... -Cheat sheet rendering completed. - -Review generated config files under output/workspaces/ -``` - -``` -~/rw-cli-codecollection/codebundles/azure-aks-triage/.test $ tree -. -├── output -│ ├── resource-dump.yaml -│ └── workspaces -│ └── [workspace-name] -│ ├── slxs -│ │ └── ac1-aa-az-aks-triage-ac6325d3 -│ │ ├── runbook.yaml -│ │ ├── sli.yaml -│ │ └── slx.yaml -│ └── workspace.yaml -├── README.md -├── Taskfile.yaml -└── workspaceInfo.yaml - - -``` - -### Uploading SLXs to the RunWhen Platform - -Most Taskfiles also include the option to `upload-slxs` and `delete-slxs`once the SLX configuration files have been reviewed for accuracy, they can be uploaded to the RunWhen Platform for further testing. - -## RunWhen Platform Workspace Testing - -Once the configuration data has been reviewed, they can be uploaded to a RunWhen Platform workspace if the `upload-slxs` and `delete-slxs` tasks exist in your Taskfile, and the following environment variables have been set: - -* RW\_API\_URL="papi.beta.runwhen.com" -* RW\_WORKSPACE="\[your-runwhen-platform-workspace-name]" -* RW\_PAT="\[your-personal-access-token]" - -``` -~/rw-cli-codecollection/codebundles/azure-aks-triage/.test$ task upload-slxs -Uploading SLX: ac1-aa-az-aks-triage-ac6325d3 to https://papi.test.runwhen.com/api/v3/workspaces/t-shea-ws-03/branches/main/slxs/ac1-aa-az-aks-triage-ac6325d3 -Successfully uploaded SLX: ac1-aa-az-aks-triage-ac6325d3 to https://papi.test.runwhen.com/api/v3/workspaces/t-shea-ws-03/branches/main/slxs/ac1-aa-az-aks-triage-ac6325d3 -``` - -{% hint style="info" %} -If the upload to the RunWhen Platform fails, feel free to contact our team. There are often validation errors that might occur but which aren't easy to surface in the failure response. Most often a configProvided value is an `integer` instead of a `string`, but other errors may not be so obvoious. -{% endhint %} - -Within a few minutes, the new SLX(s) will be available in the RunWhen Workspace Map - -

Viewing a newly uploaded SLX

- -### Updating Secrets - -While the SLI will start attempting to run immediately, you may need to select the "Edit" option on the SLX to update the Secrets configuration. In many cases, you may have Secrets configured in your Workspace for testing which won't immediately line up with the SLX configuration. - -

Editing the SLX Configuration

- -

Editing the SLI secretsProvided

- -The above screenshot outlines the secretsProvided configuration to point at a RunWhen Managed secret, which can be added/updated through **Configuration -> Secrets** - - - -### Validating Health and Tasks - -When the secrets are updated, the Health and Tasks should be tested: - -

Viewing SLI / Health

- -

Creating a RunSession with all Tasks

- -

Reviewing RunSession Report

diff --git a/docs/codecollection-development/generation-rules/README.md b/docs/codecollection-development/generation-rules/README.md deleted file mode 100644 index 8dc7a4a..0000000 --- a/docs/codecollection-development/generation-rules/README.md +++ /dev/null @@ -1,277 +0,0 @@ -# Generation Rules - -## Introduction - -The workspace builder makes use of optional **generation rules** contained in code collections to match which code bundles are relevant based on the resources discovered from indexing the configured cloud platforms. If a generation rule matches a code bundle it generates one or more SLXs for inclusion in the generated workspace. This document describes the format of generation rules and the file structure within the code collection. It is mostly of interest to code collection authors, although some of the information may be of interest to users of the workspace builder to understand how it works. - -The workspace builder operates in several phases: - -1. It scans all the code collections/bundles that are configured/enabled in the workspace info file and loads the generation rules for the enabled code bundles. -2. It indexes the resources for all the cloud platforms configured in the workspace info file. The resources are saved in a simple in-memory database. For the most part, this database doesn't really know anything about the structure or content of the resources, except for some logic that determines the parent platform-specific scope (e.g. namespace for Kubernetes, resource group for Azure). The indexing phase uses information from the generation rules about which cloud resource types are accessed to optimize the indexing process, i.e. it only indexes a resource type if there's a generation rule that accesses it. -3. The generation rules are evaluated against the discovered resources in the database and, if a match is found, one or more SLX files are scheduled to be generated. -4. It renders the file structure for the RunWhen workspace including the top-level workspace.yaml file and all the SLXs that were matching during the generation rule evaluation. - -## Directory Structure - -The generation rules are contained within the directory of its associated code bundle. A generation rule has a YAML file containing the matching logic as well as information about which SLXs to generate if a match is found. The generation of the SLX files is done with template files, so there are also one or more template files associated with the generation rule. - -Within the code bundle directory there's a directory named `.runwhen` that's the parent directory for all the generation rules for the code bundle. Under the ".runwhen" directory there's a directory named `generation-rules` that contains all the generation rule YAML files. Also under the `.runwhen` directory is a directory named `templates` that contains the SLX template files associated with the generation rules. - -``` -k8s-namespace-healthcheck# tree .runwhen -.runwhen -├── generation-rules -│   └── k8s-namespace-healthcheck.yaml -└── templates - ├── k8s-namespace-healthcheck-sli.yaml - ├── k8s-namespace-healthcheck-slo.yaml - ├── k8s-namespace-healthcheck-slx.yaml - ├── k8s-namespace-healthcheck-taskset.yaml - └── k8s-namespace-healthcheck-workflow.yaml - -``` - -See [kubernetes-generation-rule-examples.md](kubernetes-generation-rule-examples.md "mention") for more details on the specific content shown above. - -## Generation Rules - -Generation rules files are YAML files that contains one or more generation rules. Each generation rule has one or more boolean match predicates that match against properties of the resources that were discovered from indexing the configured cloud platforms. If a match is found, then the generation rule has information about which SLXs to generate and which template file to use for each SLX. - -The top-level structure of a generation rule file is: - -```yaml -apiVersion: runwhen.com/v1 -kind: GenerationRules -spec: - platform: - generationRules: - - resourceTypes: - - - matchRules: - - # match predicate specification - slxs: - - # SLX generation information -``` - -Each generation rule is targeted at a specific cloud platform. The target platform can be specified at the top level in the spec (as in the example above) or individually for each generation rule, which is useful if the generation rule targets multiple cloud platforms, although that's not a very common case. - -The currently supported platforms are: - -| Platform | Name/ID in the Generation Rulel | -| --------------------- | ------------------------------- | -| Kubernetes | kubernetes | -| Azure | azure | -| Google Cloud Platform | gcp | - -The name/id values are case-sensitive, so they must be all lower-case. The default platform is Kubernetes (mostly for historical, backward-compatibility reasons), so that's what is used if there's no explicit platform specification in the generation rule. - -### Resource Types - -Each generation rule is evaluated against a set of resource types for the specified cloud platform, specified with the resourceType field. The workspace builder iterates over all the instances of the specified resource types and evaluates each match rule against each instance looking for matches. There can be multiple resource types for a generation rule, atlhough more commonly there is just one. The resource types are platform-specific. For all the CloudQuery-based platforms the resource type is the CloudQuery table name described in the documentation for each CloudQuery source plugin, so consult the CloudQuery documentation for the relevant resource types for those platforms. For some commonly-access resources there may be a more user-friendly alias for the CQ table name. TODO: Need more info about the supported aliases as well as info about the Kubernetes resource types - -### Match Rules - -A match rule is a YAML structured object representing a boolean predicate expression. It operates primarily by performing text matches against properties of the discovered resources. Additionally, predicates can be combined using the usual compound boolean operations. - -There are two type of leaf match predicates, a "pattern" predicate and an "exists" predicate. The "pattern" predicate is used to test against various properties of the matched resource. The pattern predicate has the following fields: - -| Field Name | Description | -| ------------ | ----------------------------------------------------- | -| type | value is "pattern" for a pattern expression | -| resourceType | Specification of which resource type to match against | -| properties | List of resource properties to match against | -| pattern | Regular expression pattern to match | -| mode | String match mode, either "exact" or "substring" | - -The optional "resourceType" field is used to override the top-level resource type list, so that you can have compound match rules that match against multiple resource types. Its values are the same resource type values available for the top-level resourceType field. - -The "properties" field is a list of resource properties to match against. The following values are supported: - -| Property Name | Description | Platform | -| --------------- | ----------------------------------------------- | ---------- | -| name | The name of the resource | All | -| | Path resolved against the raw resource data | All | -| labels | All label keys and values for the resource | Kubernetes | -| label-keys | All label keys for the resource | Kubernetes | -| label-values | All label values for the resource | Kubernetes | -| annotations | All annotation keys and values for the resource | Kubernetes | -| annotation-keys | All annotation keys for the resource | Kubernetes | -| label-values | All annotation values for the resource | Kubernetes | -| namespace | Parent namespace for the resource | Kubernetes | -| cluster | Cluster for the resource | Kubernetes | -| tags | All tag keys and values for the resource | Azure | -| tag-keys | All tag keys for the resource | Azure | -| tag-values | All tag values for the resource | Azure | -| resource\_group | Parent resource group for the resource | Azure | -| project | Parent project for the resource | GCP | - -If a value in the properties list doesn't match one of these built-in values, then it is interpreted as a path expression into the raw resource data. The path expression is a slash-separated list of path components in the resource data. If one of the components in the path is a list, then the rest of the path is evaluated against each of the elements in the list and if any of them matches, then it's considered a match. - -The "pattern" field is a regular expression (Python regular expression syntax) that is matched against the specified property values. - -The "mode" field specified whether to do an exact match check or a substring check. The default value is "substring". - -An "exists" predicate is similar to a pattern predicate except that it just check for the existence of a property instead of performing a regular expression match against the value of the property. It has basically the same fields as the pattern predicate, except that there's no "pattern" or "mode" fields. It's "type" value is "exists". - -The supported compound boolean operations are AND, OR, and NOT. - -The boolean AND operation is supported with an "and" predicate type. The predicate evaluates to true only if all the child match predicates evaluate to true. The format is: - -| Field Name | Description | -| ---------- | ------------------------------------------ | -| type | Value is "and" for a boolean AND operation | -| matches | List of child match predicates | - -The boolean OR operation is supported with an "or" predicate type. The predicate evaluates to true if any of the child match predicates evaluates to true. The format is: - -| Field Name | Description | -| ---------- | ---------------------------------------- | -| type | Value is "or" for a boolean OR operation | -| matches | List of child match predicates | - -The boolean NOT operation is supported with a "not" predicate type. The predicate evaluates to true if the child predicate evaluates to false. The format is: - -| Field Name | Description | -| ---------- | ------------------------------------------ | -| type | Value is "not" for a boolean NOT operation | -| predicate | Child match predicate | - -### SLX Generations - -The list of SLXs to generate is specified in the "slxs" list. - -The SLX element has the following fields: - -| Field Name | Description | -| --------------------- | ----------------------------------------------------------------------------- | -| base\_name | The base name for the SLX | -| qualifiers | Optional list of attributes from the match resource to qualify the SLX name | -| shortened\_base\_name | Optional shortened version of the base name | -| level\_of\_detail | The level of detail for the SLX; value should be either "basic" or "detailed" | -| base\_template\_name | Base name for the SLX templates | -| output\_items | List of SLX file variants to generate | - -There are several different names associated with an SLX. The **full name** is constructed by appending the base\_name field with the values of the specified qualifier values for the matching resource, with the different components separated by dashes. - -The content of the SLX files themselves conform to the Kubernetes resource format and are treated internally by the platform as Kubernetes custom resources. The **name** of a Kubernetes custom resource is limited to 64 characters. Also, in the current implementation of the RunWhen platform these names must be unique across all instances of that SLX resource type, so the workspace name is included in the SLX resource to avoid name collisions across workspaces. Due to the 64 character limit and the inclusion of the workspace name, there are a limited number of characters left to represent the rest of the SLX name. This is usually shorter than the length of the full name of the SLX described above, so a name shortening process is performed to reduce the full name of the SLX to fit in the available length. This basically does some shortening heuristics on the individual components (i.e. base name and qualifiers) of the full SLX name (e.g. removing vowels, truncating if necessary) and then recombines them and prepends the workspace name. The shortening process also operates on the base\_name for the SLX, which eventually must be 15 characters or shorter. If you don't like the way it gets shortened heuristically and want a more human-readable full base name, you can specify the shortened\_base\_name field which is just a custom shortened version of the full base\_name (and should be 15 characters or shorter). - -The qualifiers are attributes of the matching resource that are used to ensure unique SLX names across different resource instances, typically the scoping attributes for the resource. - -The supported values are: - -| Qualifier Name | Description | Platform | -| --------------- | ----------------------------- | ---------- | -| resource | Name of the matching resource | All | -| namespace | Parent namespace | Kubernetes | -| cluster | Parent cluster | Kubernetes | -| resource\_group | Parent resource group | Azure | -| project | Parent project ID | GCP | - -In most cases the SLX name should include at least the qualifier for the platform-specific scoping construct. The exception would be if there's an SLX that should be a singleton instance across the entire target application. - -The level of detail value (described in more detail in the user reference documentation) indicates whether the SLX is basic or detailed. If the level of detail for the SLX is "basic" it will be generated for any platform scope that's configured with a level of detail value of either "basic" or "detailed". If the SLX level of detail value is "detailed" it will only be generated for platform scopes whose level of detail value is "detailed". - -The output\_items list specifies one or more SLX files to be emitted. Each entry in the list is an output item specification object with the following fields: - -| Field Name | Description | -| ------------------- | ------------------------------------------------------------------------------------ | -| type | Type of SLX file | -| path | Custom path for the emitted SLX file | -| template\_name | Name of the template file for the SLX file contents | -| template\_variables | Dictionary/Object of additional template variables to make available to the template | -| level\_of\_detail | Level of detail override for this SLX file | - -The only required field is the type field, which specifies which type of SLX file it is. The supported values are: - -| Type | Description | -| ------- | ----------------------- | -| slx | Service Level X | -| sli | Service Level Indicator | -| slo | Service Level Objective | -| runbook | Runbook | - -The other fields are optional and are typically omitted except for some special cases where it's necessary to override the default values assigned to them. - -The "path" field is the path in the workspace directory structure. It defaults to be a file in the emitted SLX directory with a YAML file name derived from the type, e.g. slx.yaml, runbook.yaml, etc. - -The template\_name is the name of the template file from the template directory to use to generate the content for the SLX file. It defaults to a name that combines the base\_template\_name field from the parent SLX specification with the type field, separated by a dash: "\-.yaml". As long as you conform to this naming convention for the template files you don't need to specify this field. - -The template\_variables field is a dictionary whose keys are the names of the additional template variables. The value of the template variable is evaluated as jinja2 template text, so it can include any of the template variables listed in the next section that discusses the SLX template files. It can also access other custom template variables that are defined before it in the dictionary. Again, this is supported for a few special cases, but will rarely be needed for typical SLXs. - -The "level\_of\_detail" field lets you override the parent SLX level\_of\_detail value for different SLX files. So, for example, if you want to have a runbook file with a "basic" level of detail and an SLI file with a "detailed" level of detail. - -### SLX Template Files - -SLX template files use the Python jinja2 format, so consult the jinja2 documentation (https://jinja.palletsprojects.com/en/3.1.x/) for info about the syntax for variable substitutions, control flow statements, etc.. The templates are rendered with the trim\_blocks and lstrip\_blocks flags enabled, which mostly just means that you can put conditional and loop statements on their own line without affecting the indentation of the text that is actually rendered, which is important to preserve the correct indentation/structure for the rendered YAML text. You can check some examples in the RunWhen-authored code collections for examples of what this looks like. - -There's a fair bit of boilerplate header content for SLX files, especially with tags and annotations, some of which is really determined by the specific version of the workspace builder that's running and not by the generation rule author. To handle this case there are some standard template files that should be included in the standard header text for the SLX. The standard SLX header content is: - -```yaml -apiVersion: runwhen.com/v1 -kind: ServiceLevelX -metadata: - name: {{slx_name}} - labels: -
- - annotations: - - -
-spec: - # The rest of the SLX content -``` - -You should basically always start an SLX template with this content. The only variation would be to add other SLX-specific labels and/or annotations before or after the inclusion of the common ones. Again, you can look at the code bundles in one of the standard RunWhen code collections for examples. - -The template will be mostly literal text, but with some variable accesses to include information specific to the matching resource. The supported template variables that are available to the template are: - -| Variable Name | Description | Supported Platforms | -| ---------------------------- | ---------------------------------------------------------------- | ------------------- | -| workspace | Name of the parent workspace | All | -| slxs\_path | Path to the "slxs" directory in the workspace | All | -| slx\_directory\_path | Path to the parent SLX directory in the workspace | All | -| slxs\_path | Path to the "slxs" directory in the workspace | All | -| default\_location | Default location value specified in the workspace info | All | -| base\_name | Base name of the SLX configured in the generation rule | All | -| slx\_name | Shortened SLX name to fit the 64 character Kubernetes name limit | All | -| full\_slx\_name | The full, unshortened SLX name | All | -| match\_resource | The matching resource instance in the resource database | All | -| custom | Object mapped from the "custom" field in the workspace info | All | -| repo\_url | Repo URL of parent code collection | All | -| ref | Git branch/tag/ref for the code collection | All | -| generation\_rule\_file\_path | Path to the generation rule that triggered the SLX generation | All | -| is\_workflow | True if a workflow file is being rendered | All | -| namespace | Parent Kubernetes namespace resource instnace | Kubernetes only | -| resource-group | Parent Azure resource group resource instance | Azure only | -| project | Parent GCP project resource instance | GCP only | - -These template variables can be used either with template variable expansion (with the \{{variable-name\}} jinja2 syntax) or as expressions in jinja2 conditional/loop statements. More information about how template variables are accessed and used can be found in the jinja2 documentation. - -Note that many of these variables are things that are used in the boilerplate header content (and the included common labels/annotations), but won't commonly be accessed by generation rule authors, so they're included mainly for reference purposes. The most commonly accessed variables are the default\_location, match\_resource, and the platform-specific scope name variables. - -The value for the match\_resource variable is the resource instance in the resource database. A resource instance has these fields: - -| Field Name | Description | Platforms | -| --------------- | ----------------------------------------------- | ---------- | -| name | Simple name. Not guaranteed to be unique | All | -| qualified\_name | Qualified name. Guaranteed to be unique | All | -| resource\_type | Prent resource type instance | All | -| resource | Raw data for the resource | All | -| labels | Label dictionary metadata for the resource | Kubernetes | -| annotations | Annotation dictionary metadata for the resource | Kubernetes | -| tags | The tag metadata for the resource | Azure | - -The simple name is not guaranteed to be unique across all the instances of a particular resource type, but the qualified name is guaranteed to be unique. - -The resource\_type field refers to a resource type instance in the resource database. The resource type instance has a field named "platform", which is a reference to a platform type in the resource database. That instance has a "name" field, which is the name of the platform. - -Again, many of these things are mostly used internally by workspace builder and won't commonly be used by generation rule authors. The most commonly accessed field will be the "resource" field with trailing field accesses to resolve to specific field in the raw resource data. So for example: - -``` - some-custom-field: {{match_resource.resource.data.foo}} -``` - -The structure of the raw resource data is platform and resource type dependent. For CloudQuery-based indexers you can consult the associated CloudQuery source plugin documentation for the supported columns for a particular table/resource-type. Each column maps to a top-level field in the raw resource data. - -The workspace builder also dumps the state of the discovered resources to a file named "resource-dump.yaml" in the shared directory. This dump is useful for learning what the structure of the raw resource data looks like and then figuring out which fields to use in the match predicates and templates. diff --git a/docs/codecollection-development/generation-rules/aws-generation-rule-examples.md b/docs/codecollection-development/generation-rules/aws-generation-rule-examples.md deleted file mode 100644 index dd57902..0000000 --- a/docs/codecollection-development/generation-rules/aws-generation-rule-examples.md +++ /dev/null @@ -1,3 +0,0 @@ -# AWS Generation Rule Examples - -Content coming soon diff --git a/docs/codecollection-development/generation-rules/azure-generation-rule-examples.md b/docs/codecollection-development/generation-rules/azure-generation-rule-examples.md deleted file mode 100644 index ea7786c..0000000 --- a/docs/codecollection-development/generation-rules/azure-generation-rule-examples.md +++ /dev/null @@ -1,3 +0,0 @@ -# Azure Generation Rule Examples - -Content coming soon diff --git a/docs/codecollection-development/generation-rules/gcp-generation-rule-examples.md b/docs/codecollection-development/generation-rules/gcp-generation-rule-examples.md deleted file mode 100644 index 4714852..0000000 --- a/docs/codecollection-development/generation-rules/gcp-generation-rule-examples.md +++ /dev/null @@ -1,3 +0,0 @@ -# GCP Generation Rule Examples - -Content coming soon diff --git a/docs/codecollection-development/generation-rules/kubernetes-generation-rule-examples.md b/docs/codecollection-development/generation-rules/kubernetes-generation-rule-examples.md deleted file mode 100644 index be9b2b9..0000000 --- a/docs/codecollection-development/generation-rules/kubernetes-generation-rule-examples.md +++ /dev/null @@ -1,527 +0,0 @@ -# Kubernetes Generation Rule Examples - -The examples below will illustrate different configurations of Generation Rules and Templates. In general, every Generation Rule should render one or more templates, which directly correspond to objects that are optional in the RunWhen platform. Every Generation Rule **must** create an SLX (which is the parent object), and will render _**one or more**_ SLI, SLO, Runbook, or Workflow - -## Simple Example - Match Every Namespace - -### Generation Rule - -In this example, the generation rule is configured match and render the appropriate templates for _**every namespace found in every cluster**_. - -``` -apiVersion: runwhen.com/v1 -kind: GenerationRules -spec: - platform: kubernetes # Kubernetes is the default when platform is not specified - generationRules: - - resourceTypes: - - namespace # Match Kubernetes Namespaces - matchRules: - - type: pattern - pattern: ".+" # Match any pattern - properties: [name] # Use the name for matching - mode: substring - slxs: - - baseName: ns-health # Use this when generating SLX names - levelOfDetail: basic # Even when doing basic scans, include this resource - qualifiers: ["namespace", "cluster"] # Generate 1 SLX for every namespace found in every cluster - baseTemplateName: k8s-namespace-healthcheck # Use this prefix looking for files in the templates folder - outputItems: - - type: slx - - type: sli - - type: slo - - type: runbook - templateName: k8s-namespace-healthcheck-taskset.yaml - - type: workflow -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-namespace-healthcheck/.runwhen/generation-rules/k8s-namespace-healthcheck.yaml) - -### SLX Template - -So, in the above example, we will generate an SLX for every namespace found in every cluster, and for each SLX, we will render the slx, sli, slo, runbook, and workflow templates found in the `templates` directory. The complete `.runwhen`directory is as follows: - -``` -k8s-namespace-healthcheck# tree .runwhen -.runwhen -├── generation-rules -│   └── k8s-namespace-healthcheck.yaml -└── templates - ├── k8s-namespace-healthcheck-sli.yaml - ├── k8s-namespace-healthcheck-slo.yaml - ├── k8s-namespace-healthcheck-slx.yaml - ├── k8s-namespace-healthcheck-taskset.yaml - └── k8s-namespace-healthcheck-workflow.yaml -``` - -Each file in the template directory is a jinja template that supports substitution from the resource that was discovered. For example, the SLX template is as follows: - -``` -apiVersion: runwhen.com/v1 -kind: ServiceLevelX -metadata: - name: {{slx_name}} - ### ^Always add this - it is auto generated. - - labels: - {% raw %} -{% include "common-labels.yaml" %} - ### ^Always add this - - annotations: - {% include "common-annotations.yaml" %} -{% endraw %} - ### ^Always add this - -spec: - imageURL: https://storage.googleapis.com/runwhen-nonprod-shared-images/icons/kubernetes/resources/labeled/ns.svg - alias: {{namespace.name}} Namespace Health - ### ^A useful name that helps identify the resource. - - asMeasuredBy: Aggregate score based on Kubernetes API Server queries - ### ^Paired with the statement, describes how the statement should be measured. - - configProvided: - - name: OBJECT_NAME - value: {{match_resource.resource.metadata.name}} - ### ^This is sometimes extra metadata to attach to an object. - - owners: - - {{workspace.owner_email}} - ### ^Always add this. This is the contact for escalations on the specific SLX. - - statement: Overall health for {{namespace.name}} should be 1, 99% of the time. # A helpful statement for someone viewing the SLX to understand more about it's desired health or automation scripts. - ### ^Paired with the asMeasureBy, this often describes the obejective of the SLX. - - additionalContext: - ### ^Additional context can be used for debuggging or Engineering Assistant search enrichment. - namespace: "{{match_resource.resource.metadata.name}}" - labelMap: "{{match_resource.resource.metadata.labels}}" - cluster: "{{ cluster.name }}" - context: "{{ cluster.context }}" - subscription_id: "{{ cluster.subscription_id }}" -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-namespace-healthcheck/.runwhen/templates/k8s-namespace-healthcheck-slx.yaml) - -A fully rendered example of the above template is as follows: - -``` -apiVersion: runwhen.com/v1 -kind: ServiceLevelX -metadata: - name: b-sandbox--ob-grnsucsc1c-ns-health-0230f7f7 - labels: - slx: b-sandbox--ob-grnsucsc1c-ns-health-0230f7f7 - workspace: b-sandbox - annotations: - fullSlxName: online-boutique-gke-runwhen-nonprod-sandbox-us-central1-sandbox-cluster-1-cluster-ns-health-0230f7f7 - sourceGenerationRuleRepoURL: https://github.com/runwhen-contrib/rw-cli-codecollection.git - sourceGenerationRuleRepoRef: main - sourceGenerationRulePath: codebundles/k8s-namespace-healthcheck/.runwhen/generation-rules/k8s-namespace-healthcheck.yaml - config.runwhen.com/last-updated-by: workspace-builder - qualifiers: '{''namespace'': ''online-boutique'', ''cluster'': ''gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster''}' -spec: - imageURL: https://storage.googleapis.com/runwhen-nonprod-shared-images/icons/kubernetes/resources/labeled/ns.svg - alias: online-boutique Namespace Health - asMeasuredBy: Aggregate score based on Kubernetes API Server queries - configProvided: - - name: OBJECT_NAME - value: online-boutique - owners: - - jarod@runwhen.com - statement: Overall health for online-boutique should be 1, 99% of the time. - additionalContext: - namespace: online-boutique - labelMap: '{''kubernetes.io/metadata.name'': ''online-boutique'', ''kustomize.toolkit.fluxcd.io/name'': - ''flux-system'', ''kustomize.toolkit.fluxcd.io/namespace'': ''flux-system''}' - cluster: gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster - context: gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster - subscription_id: missing_workspaceInfo_custom_variable - -``` - - - -## Complex Example - Match HTTPs Ingress - -### Generation Rule - -In this example, the generation rule is configured match and render the appropriate templates for _**every Kubernetes Ingress object with TLS enabled**_. - -``` -apiVersion: runwhen.com/v1 -kind: GenerationRules -spec: -platform: kubernetes # Kubernetes is the default when platform is not specified - generationRules: - - resourceTypes: - - ingress # Match Ingress objects - matchRules: - - type: and # Match when both conditions below are met - matches: - - type: pattern - pattern: ".+" - properties: [name] # Match on any name - mode: substring - - type: pattern - pattern: ".+" - properties: [spec/tls/hosts] # Match on this field with any content - mode: substring - slxs: - - baseName: http-ok-tls-test - qualifiers: ["resource", "namespace", "cluster"] # Generate 1 SLX for every ingress object found in every namespace in every cluster. - baseTemplateName: http-ok-tls - levelOfDetail: basic # Even when doing basic scans, include this resource - outputItems: - - type: slx - - type: sli - - type: slo - - type: runbook - templateName: http-ok-tls-taskset.yaml -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/curl-http-ok/.runwhen/generation-rules/http-ok-tls.yaml) - -### SLI Template - -In the previous example, we outlined an SLX template that was generated for every namespace found in every cluster. In this example, we will highlight the SLI template that is found in the `templates` directory. The \*complete `.runwhen`directory is as follows: - -``` -.runwhen -├── generation-rules -│   ├── http-ok-tls.yaml -└── templates - ├── http-ok-tls-sli.yaml - ├── http-ok-tls-slo.yaml - ├── http-ok-tls-slx.yaml - └── http-ok-tls-taskset.yaml - -``` - -{% hint style="info" %} -\*Note: The [curl-http-ok CodeBundle](https://github.com/runwhen-contrib/rw-cli-codecollection/tree/main/codebundles/curl-http-ok)[ ](https://registry.runwhen.com/CodeCollection/rw-cli-codecollection/curl-http-ok/health/)is a good example of having multiple generation rules and templates that leverage the same CodeBundle, but with differing configuration templates. View all generation rules for this CodeBundle at [this link](https://github.com/runwhen-contrib/rw-cli-codecollection/tree/main/codebundles/curl-http-ok/.runwhen/generation-rules). -{% endhint %} - -Since the previous example shows an SLX, this example will outline the SLI template: - -``` -apiVersion: runwhen.com/v1 -kind: ServiceLevelIndicator -metadata: - name: {{slx_name}} - ### ^Always add this - it is auto generated. - - labels: - {% raw %} -{% include "common-labels.yaml" %} - ### ^Always add this - - annotations: - {% include "common-annotations.yaml" %} - ### ^Always add this - -spec: - displayUnitsLong: OK - ### ^How the chart units should be displayed (long form) - - displayUnitsShort: ok - ### ^How the chart units should be displayed (short form) - - locations: - - {{default_location}} - ### ^Always add this - - description: Measures the response code and latency to the ingress object. - ### ^Describe what the SLI measures. - - codeBundle: - {% if repo_url %} - repoUrl: {{repo_url}} - {% else %} - repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git - ### ^The default url to the CodeCollection. - - {% endif %} - {% if ref %} - ref: {{ref}} - {% else %} - ref: main - {% endif %} -{% endraw %} - pathToRobot: codebundles/curl-http-ok/sli.robot - ### ^The default path to the CodeBundle code - - ### ^The jijna if/else above is used to support user overrides for forks/branches/etc. - - intervalStrategy: intermezzo - intervalSeconds: 30 - ### ^How often to run the SLI code. - - configProvided: - - name: URL - value: https://{{match_resource.resource.spec.tls[0].hosts[0]}} - ### ^The rendered value from the discovered/matched resource. - - - name: TARGET_LATENCY - value: '1.2' - ### ^A sensible default for the code to function. - - - name: DESIRED_RESPONSE_CODE - value: '200' - ### ^A sensible default for the code to function. - - secretsProvided: [] -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/curl-http-ok/.runwhen/templates/http-ok-tls-sli.yaml) - -A fully rendered example of the above template is as follows: - -``` -apiVersion: runwhen.com/v1 -kind: ServiceLevelIndicator -metadata: - name: b-sandbox--sas-tt-grnsucsc1c-http-ok-tls-aks-c4cf8a15 - labels: - slx: b-sandbox--sas-tt-grnsucsc1c-http-ok-tls-aks-c4cf8a15 - workspace: b-sandbox - annotations: - fullSlxName: sample-app-service-trouble-town-gke-runwhen-nonprod-sandbox-us-central1-sandbox-cluster-1-cluster-http-ok-tls-aks-c4cf8a15 - sourceGenerationRuleRepoURL: https://github.com/runwhen-contrib/rw-cli-codecollection.git - sourceGenerationRuleRepoRef: main - sourceGenerationRulePath: codebundles/curl-http-ok/.runwhen/generation-rules/http-ok-tls-aks-public-loadbalancer-ext-dns-tls.yaml - config.runwhen.com/last-updated-by: workspace-builder - qualifiers: '{''resource'': ''sample-app-service'', ''namespace'': ''trouble-town'', - ''cluster'': ''gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster''}' -spec: - displayUnitsLong: OK - displayUnitsShort: ok - locations: - - location-01-us-west1 - description: Measures the response code and latency to the AKS LoadBalancer Object. - codeBundle: - repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git - ref: main - pathToRobot: codebundles/curl-http-ok/sli.robot - intervalStrategy: intermezzo - intervalSeconds: 30 - configProvided: - - name: URL - value: https://sampleapp.example.com - - name: TARGET_LATENCY - value: '1.2' - - name: DESIRED_RESPONSE_CODE - value: '200' - secretsProvided: [] -``` - - - -## Complex Example - Match Specific Deployments & Use Secrets - -### Generation Rule - -In this final example, the generation rule is configured match and render the appropriate templates for _**every Kubernetes Deployment with a specific label AND a default-container configured**_. - -``` -apiVersion: runwhen.com/v1 -kind: GenerationRules -spec: - generationRules: - - resourceTypes: - - deployment # Match Kubernetes Deployments - matchRules: - - type: and # Require both conditions to be satisfied - matches: - - type: pattern - pattern: ".+" - properties: ["spec/template/metadata/annotations/kubectl.kubernetes.io//default-container"] - mode: substring - ### ^The above configuration MUST exist on the deployment. - - - type: pattern - pattern: "codecollection.runwhen.com/app" - properties: [labels] - mode: substring - ### ^The above label MUST exist on the deployment. - - slxs: - - baseName: k8s-tail-logs-dynamic - qualifiers: ["resource", "namespace", "cluster"] # Generate 1 SLX for every (matched) deployment found in every namespace in every cluster. - baseTemplateName: k8s-tail-logs-dynamic - levelOfDetail: detailed - outputItems: - - type: slx - - type: slo - - type: runbook - templateName: k8s-tail-logs-dynamic-taskset.yaml - - type: sli -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-tail-logs-dynamic/.runwhen/generation-rules/k8s-tail-logs-dynamic.yaml) - -### Runboook/TaskSet Example - -{% hint style="info" %} -Runbooks and TaskSets are terms that are used interchangeably. The CodeBundle code must exist in a file called runbook.robot, but we sometimes refer to this runbook.robot code as TaskSets. -{% endhint %} - -n the previous example, we outlined an SLI template that was generated for every ingress object found in every namespace & every cluster. In this example, we will highlight the Runbook/TastkSet template that is found in the `templates` directory. The \*complete `.runwhen`directory is as follows: - -``` -.runwhen -├── generation-rules -│   └── k8s-tail-logs-dynamic.yaml -└── templates - ├── k8s-tail-logs-dynamic-sli.yaml - ├── k8s-tail-logs-dynamic-slo.yaml - ├── k8s-tail-logs-dynamic-slx.yaml - └── k8s-tail-logs-dynamic-taskset.yaml -``` - -Since the previous example shows an SLI, this example will outline the Runbook/TaskSet template: - -``` -apiVersion: runwhen.com/v1 -kind: Runbook -metadata: - name: {{slx_name}} - ### ^Always add this - it is auto generated. - - labels: - {% raw %} -{% include "common-labels.yaml" %} - ### ^Always add this - - annotations: - {% include "common-annotations.yaml" %} - ### ^Always add this - -spec: - location: {{default_location}} - codeBundle: - {% if repo_url %} - repoUrl: {{repo_url}} - {% else %} - repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git - ### ^The default url to the CodeCollection. - - {% endif %} - {% if ref %} - ref: {{ref}} - {% else %} - ref: main - {% endif %} - pathToRobot: codebundles/k8s-tail-logs-dynamic/runbook.robot - ### ^The default path to the CodeBundle code - - ### ^The jijna if/else above is used to support user overrides for forks/branches/etc. - - configProvided: - - name: KUBERNETES_DISTRIBUTION_BINARY - value: kubectl - ### ^A sensible default for the code to function. - - - name: LOGS_SINCE - value: 10m - ### ^A sensible default for the code to function. - - - name: LABELS - value: codecollection.runwhen.com/app={{match_resource.resource.metadata.labels.get('codecollection.runwhen.com/app')}} - ### ^The rendered value from the discovered/matched resource for the specific labels to use. - - - name: EXCLUDE_PATTERN - value: INFO - - name: CONTAINER_NAME - value: {{match_resource.resource.spec.template.metadata.annotations.get('kubectl.kubernetes.io/default-container')}} - ### ^The rendered value from the discovered/matched resouce for the default container. - - - name: MAX_LOG_LINES - value: '500' - - name: NAMESPACE - value: {{match_resource.resource.metadata.namespace}} - ### ^The rendered value from the discovered/matched resource. - - - name: CONTEXT - value: {{context}} - ### ^The rendered value from the discovered/matched resource. - - - name: STACKTRACE_PARSER - value: Dynamic - ### ^A sensible default for the code to function. - - - name: INPUT_MODE - value: SPLIT - ### ^A sensible default for the code to function. - - - name: MAX_LOG_BYTES - value: '2560000' - ### ^A sensible default for the code to function. - - secretsProvided: - {% if wb_version %} - {% include "kubernetes-auth.yaml" ignore missing %} - {% else %} - - name: kubeconfig - workspaceKey: {{custom.kubeconfig_secret_name}} - {% endif %} -{% endraw %} - ### ^Use this when the secrets needed are for Kubernetes. These templates help - ### substitute the right authentcation for the users environment. -``` - -[Reference example](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/k8s-tail-logs-dynamic/.runwhen/templates/k8s-tail-logs-dynamic-taskset.yaml) - -A fully rendered example of the above template is as follows: - -``` -apiVersion: runwhen.com/v1 -kind: Runbook -metadata: - name: b-sandbox--v-va-grnsucsc1c-k8s-tl-lgs-dyn-8bc522d2 - labels: - slx: b-sandbox--v-va-grnsucsc1c-k8s-tl-lgs-dyn-8bc522d2 - workspace: b-sandbox - annotations: - fullSlxName: vote-voting-app-gke-runwhen-nonprod-sandbox-us-central1-sandbox-cluster-1-cluster-k8s-tail-logs-dynamic-8bc522d2 - sourceGenerationRuleRepoURL: https://github.com/runwhen-contrib/rw-cli-codecollection.git - sourceGenerationRuleRepoRef: main - sourceGenerationRulePath: codebundles/k8s-tail-logs-dynamic/.runwhen/generation-rules/k8s-tail-logs-dynamic.yaml - config.runwhen.com/last-updated-by: workspace-builder - qualifiers: '{''resource'': ''vote'', ''namespace'': ''voting-app'', ''cluster'': - ''gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster''}' -spec: - location: location-01-us-west1 - codeBundle: - repoUrl: https://github.com/runwhen-contrib/rw-cli-codecollection.git - ref: main - pathToRobot: codebundles/k8s-tail-logs-dynamic/runbook.robot - configProvided: - - name: KUBERNETES_DISTRIBUTION_BINARY - value: kubectl - - name: LOGS_SINCE - value: 10m - - name: LABELS - value: codecollection.runwhen.com/app=vote - - name: EXCLUDE_PATTERN - value: INFO - - name: CONTAINER_NAME - value: vote - - name: MAX_LOG_LINES - value: '500' - - name: NAMESPACE - value: voting-app - - name: CONTEXT - value: gke_runwhen-nonprod-sandbox_us-central1_sandbox-cluster-1-cluster - - name: STACKTRACE_PARSER - value: Dynamic - - name: INPUT_MODE - value: SPLIT - - name: MAX_LOG_BYTES - value: '2560000' - secretsProvided: - - name: kubeconfig - workspaceKey: kubeconfig - -``` diff --git a/docs/codecollection-development/prerequisites.md b/docs/codecollection-development/prerequisites.md deleted file mode 100644 index 07670b5..0000000 --- a/docs/codecollection-development/prerequisites.md +++ /dev/null @@ -1,35 +0,0 @@ -# Requirements - -## Concepts / Resources - -Familiarity with: - -* Our [contributing doc](../cc-dev/contributing.md) -* RunWhen [Terms and Concepts](https://docs.runwhen.com/public/runwhen-platform/terms-and-concepts) -* Review our [code-of-conduct.md](../authors-program-details/code-of-conduct.md "mention") -* Google [python style guide](https://google.github.io/styleguide/pyguide.html); we generally try to follow it but do stray when needed for certain designs. -* [Robot Framework](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html), although a strong understanding of Robot is not needed. We try to handle complex logic at the python or external script level and use Robot like a configuration language, passing data and settings between Robot Tasks. - - - -## Development Environment - -Authoring and testing CodeBundles requires: - -* A Public [GitHub](https://www.github.com) Repository -* The ability to run a devcontainer image (e.g. via [GitHub Codepaces](https://github.com/features/codespaces), [VSCode](https://code.visualstudio.com/docs/devcontainers/containers)) -* Lab / test infrastructure; CodeBundles should be adequately tested against test infrastructure from the local environment prior to uploading to the RunWhen Platform Workspace for testing. -* A [RunWhen Platform Workspace](https://docs.runwhen.com/public/getting-started/creating-a-runwhen-workspace); for thorough testing, the RunWhen Platform (SaaS) service should be able to access the test infrastructure. - - - - - -## CodeCollection Development Image - -The CodeCollection development image is specified in the devcontainer.json spec and is built from the local Dockerfile inside of the CodeCollection template repository. Developers can customize this Dockerfile as needed, though it can't guarantee that tools installed in this image will function properly within the SaaS platform and a conversation with our team on Slack may be required. - -

CodeCollection Development Image Layers

- - - diff --git a/docs/codecollection-development/running-your-first-codebundle-locally.md b/docs/codecollection-development/running-your-first-codebundle-locally.md deleted file mode 100644 index f9ed0b9..0000000 --- a/docs/codecollection-development/running-your-first-codebundle-locally.md +++ /dev/null @@ -1,199 +0,0 @@ -# CodeBundle Basics - -{% hint style="info" %} -Please see [running-your-first-codebundle.md](running-your-first-codebundle.md "mention")before referencing this page. -{% endhint %} - -## Types of CodeBundles - -There are two primary CodeBundles: - -* Service Level Indicator (SLI) -* TaskSet (Runbook) - -Each of these CodeBundles is represented in a single `.robot` file, which constists of `Tasks` to perform, along with user-provided `Variables` and dependent `Libraries` that are used to support the execution of each `Task`. - -

CodeBundle Types

- -### Service Level Indicator CodeBundles - -The Service Level Indicate CodeBundle can be described as follows: - -* written in the `sli.robot` file -* runs on a scheduled basis, for example, every 30 or 60 seconds -* is primarily responsible for fetching or generating a metric that is pushed to the RunWhen Platform -* these metrics can be used to trigger alerts or assist in SLO calculations - -For example, a simple SLI CodeBundle might have a task that counts the number of unready pods in a namespace, and then pushes that total value to the RunWhen Platform. That number can then be used to trigger an alert if more than 1 pod us unready. - -### TaskSet CodeBundles - -The TaskSet CodeBundle can be describe as follows: - -* written in the `runbook.robot` file, typically with multiple Tasks -* runs on-demand, for example, when manually executed by a user, or automatically executed from an Engineering Assistant or workflow (triggered from an alert, webhook, or some other event) -* is primarily responsible for fetching information, adding details to a report, generating high-quality `Issues` and `Next Steps` for handoff to an Engineering Assistant or human escalation - -For example, related to the previous SLI use case, the TaskSet CodeBundle would have 1 Task that identifies which deployments have unready pods in the namespace, another Task might fetch a list of events related to each specific unready pod, which can then raise an `Issue` about the deployment (e.g. Deployment 'cartservice' has unready pods) with some `Next Steps` that are related to the identified events (e.g. Add resource to cluster. Could not schedule pod due to capacity limits). - -## CodeBundle Structure - -This section looks at a [basic CodeBundle](https://github.com/runwhen-contrib/rw-cli-codecollection/blob/main/codebundles/curl-http-ok/sli.robot) to review each core component. - -### Settings - -The **Settings** section specifies the documentation, metadata, and required libraries. - -* **Documentation**: Provides a brief summary of the taskset. -* **Metadata**: - * **Author**: The author's name (`stewartshea`). - * **Display Name**: The taskset’s display name (`cURL HTTP OK`). - * **Supports**: Specifies supported operating systems (`Linux`, `macOS`, `Windows`) and protocols (`HTTP`). -* **Library**: - * **BuiltIn**: Imports Robot Framework’s built-in functions. - * **RW.Core**: Imports the core RunWhen library for custom functionalities. - * **RW.CLI**: Provides CLI (Command Line Interface) capabilities within the RunWhen platform. - * **RW.platform**: Allows platform-specific interactions within the taskset. -* **Suite Setup**: Defines a suite-wide setup step to initialize variables and configurations used across tasks - -``` -*** Settings *** -Documentation This taskset uses curl to validate the response code of the endpoint. Returns ascore of 1 if healthy, an 0 if unhealthy. -Metadata Author stewartshea -Metadata Display Name cURL HTTP OK -Metadata Supports Linux macOS Windows HTTP -Library BuiltIn -Library RW.Core -Library RW.CLI -Library RW.platform - - -Suite Setup Suite Initialization -``` - -### Keywords - -#### Suite Initialization - -Initializes the suite by importing and setting up the necessary user-defined variables for this taskset. - -* **URL**: The endpoint URL to be checked. - * **Type**: `string` - * **Description**: Specifies the URL to perform HTTP requests against. - * **Default**: `https://www.runwhen.com` -* **TARGET\_LATENCY**: Maximum allowable latency (in seconds). - * **Type**: `string` - * **Description**: The maximum response time in seconds allowed for requests. - * **Default**: `1.2` -* **DESIRED\_RESPONSE\_CODE**: Expected HTTP response code indicating success. - * **Type**: `string` - * **Description**: The response code that represents a successful response. - * **Default**: `200` - -``` -*** Keywords *** -Suite Initialization - ${URL}= RW.Core.Import User Variable URL - ... type=string - ... description=What URL to perform requests against. - ... pattern=\w* - ... default=https://www.runwhen.com - ... example=https://www.runwhen.com - ${TARGET_LATENCY}= RW.Core.Import User Variable TARGET_LATENCY - ... type=string - ... description=The maximum latency in seconds as a float value allowed for requests to have. - ... pattern=\w* - ... default=1.2 - ... example=1.2 - ${DESIRED_RESPONSE_CODE}= RW.Core.Import User Variable DESIRED_RESPONSE_CODE - ... type=string - ... description=The response code that indicates success. - ... pattern=\w* - ... default=200 - ... example=200 -``` - - - -### Tasks - -A list of tasks to execute. Each task should be able to execute indepenent of each other. - -#### Checking HTTP URL Is Available And Timely - -This task validates that the specified URL is available and meets the desired response criteria. - -* **Documentation**: Uses `curl` to validate the HTTP response and check endpoint latency. -* **Tags**: `cURL`, `HTTP`, `Ingress`, `Latency`, `Errors` - -Steps: - -1. **Run cURL Command**: - * Executes a `curl` command against the URL. - * Outputs a JSON-like string with `http_code` (status code) and `time_total` (response time). -2. **Parse Response**: - * Uses `json.loads` to parse the JSON output from `curl`. - * Extracts `time_total` (latency) and `http_code` (HTTP status code). -3. **Evaluate Conditions**: - * **Latency Check**: Checks if the latency is within the target value (`TARGET_LATENCY`), returning `1` if true, otherwise `0`. - * **Status Code Check**: Validates if the HTTP status code matches the desired response code (`DESIRED_RESPONSE_CODE`), returning `1` if true, otherwise `0`. -4. **Score Calculation**: - * Combines the latency and status code checks to produce a final score. - * **Score**: `1` if both checks pass, otherwise `0`. -5. **Push Metric**: - * Pushes the resulting score to the RunWhen platform as a metric. - -``` -*** Tasks *** -Checking HTTP URL Is Available And Timely - [Documentation] Use cURL to validate the http response - [Tags] cURL HTTP Ingress Latency Errors - ${curl_rsp}= RW.CLI.Run Cli - ... cmd=curl -o /dev/null -w '{"http_code": \%{http_code}, "time_total": \%{time_total}}' -s ${URL} - ${json}= evaluate json.loads($curl_rsp.stdout) - ${latency}= Set Variable ${json['time_total']} - ${latency_within_target}= Evaluate 1 if ${latency} <= ${TARGET_LATENCY} else 0 - ${status_code}= Set Variable ${json['http_code']} - ${http_ok}= Evaluate 1 if ${status_code} == ${DESIRED_RESPONSE_CODE} else 0 - ${score}= Evaluate int(${latency_within_target}*${http_ok}) - RW.Core.Push Metric ${score} -``` - -## RunWhen Development Tools - -The devtools container image contains some development tools that help with local testing and replicate platform functionality. One such example is the ability to pull in an Environment Variable as a Secret for use with the CodeBundle. The following list of developer interfaces or environment variables can be used during local development. - -### Environment Variables - -#### RW\_FROM\_FILE - -For some secrets you may wish to store them in files rather than environment variables(json service accounts are a good example). The devtools container contains an `/~/auth/` directory ( which is gitignored) for storing secret files during development. In order to tell the environment to pull in secrets from a specific file, set the environment variable with a file mapping, such as: - -* Local Environment Configuration - -`export RW_FROM_FILE='{"kubeconfig":"/home/runwhen/auth/kubeconfig"}'` - -* Usage in a CodeBundle - -``` -*** Keywords *** -Suite Initialization - ${kubeconfig}= RW.Core.Import Secret kubeconfig - ... type=string - ... description=The kubeconfig secret to use for authenticating with the cluster. - ... pattern=\w* -``` - -#### RW\_SECRET\_REMAP - -If you're working with multiple similar secrets and wish to avoid constantly re-exporting them, you can remap them by setting this environment variable: - -`RW_SECRET_REMAP='{"kubeconfig":"MY_KUBECONFIG"}'` - -This will cause all instances of `kubeconfig` to use `MY_KUBECONFIG` when the CodeBundle imports a secret called `kubeconfig` - -#### RW\_ENV\_REMAP - -Similar to the secret name remap, you can do the same for environment variables to avoid re-exporting them: - -`RW_ENV_REMAP='{"PROMETHEUS_HOSTNAME":"PROM_HOSTNAME"}'` diff --git a/docs/codecollection-development/running-your-first-codebundle.md b/docs/codecollection-development/running-your-first-codebundle.md deleted file mode 100644 index 04f8de3..0000000 --- a/docs/codecollection-development/running-your-first-codebundle.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: >- - This guide quickly gets you up and running using the repository template, the - devtools container image, and a few basic CodeBundle tests. ---- - -# Getting Started - -## CodeCollection Development Environment Setup - -### Repository Initialization - -To get started, first create a CodeCollection using the repository [template ](https://github.com/runwhen-contrib/codecollection-template)on GitHub. Select the `Create a new repository` option from the `Use this template` dropdown. - -

Creating a CodeCollection Repository from Template

- -

Example CodeCollection Repository

- -### Starting Your Development Container In GitHub Codespaces - -With your template created you'll be able to run it in codespaces (or locally using the [devcontainer VSCode plugin](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)). - -
- -Once the devcontainer has started, your enviornment should look as follows: - -
- -#### Starting Your Development Environment in VS Code - -If running in Visual Studio Code, clone the repository and upon opening the repo, access the Command Pallet (Ctrl + Shift + P) and select "Dev Containers: Reopen in Container" - -

Opening the the devcontainer locally in vscode

- - - - - -### Validate CodeBundle Functionality - -To ensure that the environment is functioning, run the hello world CodeBundle. Use the `ro` cli utility, which is just a simple wrapper for running robot, to run the sli.robot file. - -``` -runwhen@codespaces-29ff86:~/codecollection/$ cd codebundles/hello_world -runwhen@codespaces-29ff86:~/codecollection/codebundles/hello_world$ ro sli.robot -============================================================================== -Sli :: This is a hello world codebundle! -============================================================================== -Hello World .. -Push metric: value:1 sub_name:None metric_type:untyped labels:{} - -Hello World | PASS | ------------------------------------------------------------------------------- -Sli :: This is a hello world codebundle! | PASS | -1 task, 1 passed, 0 failed -============================================================================== -Output: /robot_logs/sli-output.xml -Log: /robot_logs/sli-log.html -Report: /robot_logs/sli-report.html -runwhen@codespaces-29ff86:~/codecollection/codebundles/hello_world$ -``` - -To view the detailed log output, you can select the Ports tab and open port 3000 in the browser: - -

Accessing the HTTP Log Server

- -
- -

Viewing the HTTP Log Server Directories

- -
- -

Viewing Robot Logs

- - - diff --git a/docs/codecollection-development/sandbox-resources/README.md b/docs/codecollection-development/sandbox-resources/README.md deleted file mode 100644 index 4b58adb..0000000 --- a/docs/codecollection-development/sandbox-resources/README.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: >- - Access to the RunWhen Sandbox can help Authors to test codebundles against the - same infrastructure that RunWhen uses for testing and demos. ---- - -# Sandbox / Lab Resources - -## Sandbox Cluster - -RunWhen operates a sandbox GKE cluster that is used for testing and demonstration purposes. For codebundle authors, RunWhen provides access to this cluster to ease the burden on setting up test infrastructure. - -## Application/Infrastructure Stacks - -The Sandbox cluster provides many different application stacks, and also hosts a demo version of [RunWhen Local](https://docs.runwhen.com/public/v/runwhen-local), which provides some insight as to what is running in the cluster (see [here](https://runwhen-local.sandbox.runwhen.com/)) and what troubleshooting commands are available through the [RunWhen Local Troubleshooting Cheat Sheet](https://docs.runwhen.com/public/v/runwhen-local/user-guide/features/user\_guide-feature\_overview). - -Some of the resources include: - -* ArgoCD -* Artifactory -* Cert-Manager -* Crossplane -* External-DNS -* FluxCD -* Ingress - Kong -* Ingress - Nginx -* Jenkins (Traditional) -* [jenkinsx.md](jenkinsx.md "mention") -* MongoDB (Operator and test database) -* Online Boutique -* Otel-Demo -* Postgres (Operator and test database) -* Vault (HashiCorp) - -### Obtaining Sandbox Access - -Access to the sandbox cluster is provided by a shared kubeconfig. Reach out via [Slack](https://runwhen.slack.com/join/shared\_invite/zt-1l7t3tdzl-IzB8gXDsWtHkT8C5nufm2A#/shared-invite/email) or [Discord](https://discord.com/invite/Ut7Ws4rm8Q) to connect with us for read-only access to the cluster. - -{% hint style="info" %} -A dedicated sandbox Vault instance is coming soon for sharing secrets / credentials with codebundle authors. -{% endhint %} - -### Requesting Additional Application Stacks - -Want to write a new codebundle for an application stack that is missing from the Sandbox? Reach out via [Slack](https://runwhen.slack.com/join/shared\_invite/zt-1l7t3tdzl-IzB8gXDsWtHkT8C5nufm2A#/shared-invite/email) or [Discord](https://discord.com/invite/Ut7Ws4rm8Q) to connect with us and we can help. . diff --git a/docs/codecollection-development/sandbox-resources/jenkinsx.md b/docs/codecollection-development/sandbox-resources/jenkinsx.md deleted file mode 100644 index 6e2f247..0000000 --- a/docs/codecollection-development/sandbox-resources/jenkinsx.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -description: >- - This page outlines the JenkinsX resources that are available in the Sandbox - for developing codebundles. ---- - -# JenkinsX - -## Repositories & Collaborator Access - -Currently two repositories exist that support JenkinsX: - -* [Cluster Repository](https://github.com/runwhen-contrib/sandbox-jenkinsx-cluster) - The main GitOps repo that configures JenkinsX -* [Demo App](https://github.com/runwhen-contrib/sandbox-jenkinsx-demo-app) - A sample application that is mangaged by JenkinsX - -All codebundle authors must be added as a collaborator on the "Demo App" repository in order to effectively test JenkinsX pipeline functionality. Reach out via [Slack](https://runwhen.slack.com/join/shared\_invite/zt-1l7t3tdzl-IzB8gXDsWtHkT8C5nufm2A#/shared-invite/email) or [Discord](https://discord.com/invite/Ut7Ws4rm8Q) to connect with us to be added as a collaborator or to request additional application resources. - -### JX CLI Access - -{% hint style="success" %} -When working with multiple Kubeconfigs, you can set the KUBECONFIG environment variable in the jx command, such as `KUBECONFIG=author-kubeconfig jx pipeline ls -n jx` -{% endhint %} - -The `jx` cli should be available and accessible with the provided kubeconfig. This utility should have the necessary permissions to list or view jx and pipeline related activities, for example: - -* Listing pipeline jobs - -``` -$ jx pipeline ls -n jx -Name URL LAST_BUILD STATUS DURATION -runwhen-contrib/sandbox-jenkinsx-demo-app/main #1694005998381 completed-release N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx-quickstart-golang-http/master #1693402143058 completed-release N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx/main #1693498923664 completed-bootjob N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx/main #1693499039486 completed-bootjob N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx/main #1693499548239 completed-bootjob N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx/main #1693499828520 completed-bootjob N/A N/A N/A N/A -runwhen/infra-flux-nonprod-sandbox-jx/main #1693499939820 completed-bootjob N/A N/A N/A N/A - -``` - -* Reviewing the latest admin log - -``` -$ jx admin log -n jx -? select the Job to view: [Use arrows to move, type to filter] -> #10 started 35m0s Succeeded - #9 started 38m0s Succeeded - #8 started 49m0s Succeeded -``` - -### WebUI Access - -{% hint style="info" %} -The following interfaces may require user/password. This can be provided upon request via Slack or Discord. -{% endhint %} - -JenkinsX provides a few different interfaces that can be used instead of the jx cli. These are available via the `kubectl port-foward` command: - -* Lighthouse WebUI (port-forwarded to localhost:8082): `kubectl port-forward svc/lighthouse-webui-plugin -n jx 8082:80` - -

Lighthouse Web UI

- -* Pipeline Visualizer (port-forwarded to localhost:8081): `kubectl port-forward svc/jx-pipelines-visualizer -n jx 8081:80` - -

JenkinsX Pipeline Visualizer

- -
- -### diff --git a/docs/codecollection-development/sandbox-resources/postgres-operator-and-test-database.md b/docs/codecollection-development/sandbox-resources/postgres-operator-and-test-database.md deleted file mode 100644 index d887ab2..0000000 --- a/docs/codecollection-development/sandbox-resources/postgres-operator-and-test-database.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: >- - This page outlines the PostgreSQL resources that are available in the Sandbox - for developing codebundles. ---- - -# Postgres (Operator and test database) - -The `postgres-database` namespace contains: - -* The [Zalando Postgres Operator ](https://github.com/zalando/postgres-operator)deployed via the [Helm Chart](https://postgres-operator.readthedocs.io/en/latest/quickstart/#helm-chart) -* A minimal 2 instance postresql instance - -``` -apiVersion: "acid.zalan.do/v1" -kind: postgresql -metadata: - name: acid-minimal-cluster -spec: - teamId: "acid" - volume: - size: 1Gi - numberOfInstances: 2 - users: - zalando: # database owner - - superuser - - createdb - foo_user: [] # role for application foo - databases: - foo: zalando # dbname: owner - preparedDatabases: - bar: {} - postgresql: - version: "15" -``` - - - -The `runwhen-authors` service account has access to `get, list, watch` the resources in this namespace, along with: - -* Viewing secrets (in order to test database codebundles) -* Pod exec capabilities - -#### Viewing Pods - -``` -# kubectl get pods -n postgres-database -NAME READY STATUS RESTARTS AGE -acid-minimal-cluster-0 1/1 Running 0 13m -acid-minimal-cluster-1 1/1 Running 0 7m48s -postgres-operator-7d446d785-vhb98 1/1 Running 0 36m -``` - -#### Viewing Secrets - -``` -# kubectl get secret -n postgres-database -NAME TYPE DATA AGE -foo-user.acid-minimal-cluster.credentials.postgresql.acid.zalan.do Opaque 2 13m -postgres.acid-minimal-cluster.credentials.postgresql.acid.zalan.do Opaque 2 13m -sh.helm.release.v1.postgres-operator.v1 helm.sh/release.v1 1 36m -standby.acid-minimal-cluster.credentials.postgresql.acid.zalan.do Opaque 2 13m -zalando.acid-minimal-cluster.credentials.postgresql.acid.zalan.do Opaque 2 13m -``` - -#### Exec into the PostgreSQL Pod - -``` -# kubectl exec -it acid-minimal-cluster-0 -n postgres-database -- /bin/bash - - ____ _ _ -/ ___| _ __ (_) | ___ -\___ \| '_ \| | |/ _ \ - ___) | |_) | | | (_) | -|____/| .__/|_|_|\___/ - |_| - -This container is managed by runit, when stopping/starting services use sv - -Examples: - -sv stop cron -sv restart patroni - -Current status: (sv status /etc/service/*) - -run: /etc/service/patroni: (pid 33) 920s -run: /etc/service/pgqd: (pid 34) 920s -root@acid-minimal-cluster-0:/home/postgres# -``` diff --git a/docs/cursor-remote-devcontainer.md b/docs/cursor-remote-devcontainer.md new file mode 100644 index 0000000..fb03f5e --- /dev/null +++ b/docs/cursor-remote-devcontainer.md @@ -0,0 +1,109 @@ +# Cursor with this devcontainer (GitHub Codespaces) + +Use a **GitHub Codespace** for this repo, then attach **Cursor** with **Remote - SSH** using the **GitHub CLI** (`gh codespace ssh`). Authentication is handled by GitHub and `gh`. + +The devcontainer includes the [`sshd` feature](https://github.com/devcontainers/features/tree/main/src/sshd) so Codespaces SSH works as documented by GitHub. + +**Further reading** + +- [Connecting GitHub Codespaces to Cursor](https://medium.com/@NFAblog/connect-github-codespaces-to-cursor-ai-ai-friendly-vs-code-clone-243fa5f79414) +- [Cursor AppImage on Ubuntu 24.04](https://dev.to/melvincarvalho/how-to-fix-cursor-appimage-not-running-on-ubuntu-2404-2o3p) + +--- + +## 1. `sshd` in `devcontainer.json` + +```json +"ghcr.io/devcontainers/features/sshd:1": { + "version": "latest" +} +``` + +Rebuild the devcontainer after changing features. + +--- + +## 2. GitHub CLI + +Install [GitHub CLI](https://cli.github.com/) (`gh`). If `gh codespace ssh` misbehaves, update to a recent `gh` release. + +--- + +## 3. List codespaces and open a shell + +```bash +gh codespace list + +gh codespace ssh -c YOUR_CODESPACE_NAME -- +``` + +Use your codespace name from the list. Shell user is **`runwhen`**. + +--- + +## 4. SSH config for Cursor + +```bash +gh codespace ssh -c YOUR_CODESPACE_NAME --config >> ~/.ssh/codespaces +``` + +Edit the new **`Host`** block: + +- Remove **`IdentityFile …/codespaces.auto`** +- In **`ProxyCommand`**, remove **`-i …/codespaces.auto`**; keep **`gh cs ssh … --stdio --`** (end with `--` only) + +Example: + +```sshconfig +Host cs.YOUR_CODESPACE_NAME.remote-updates + User runwhen + ProxyCommand ... gh cs ssh -c YOUR_CODESPACE_NAME --stdio -- -i ~/.ssh/codespaces.auto + IdentityFile ~/.ssh/codespaces.auto + ... +``` + +After editing, **`ProxyCommand`** should end with **`--stdio --`** and there should be **no** `IdentityFile` for `codespaces.auto`. + +Bulk cleanup example: + +```bash +sed -i.bak \ + -e 's|IdentityFile .*/\.ssh/codespaces\.auto||g' \ + -e 's|-i .*/\.ssh/codespaces\.auto||g' \ + ~/.ssh/codespaces +``` + +In Cursor: **Remote-SSH: Connect to Host…** → select that host → **Open Folder** → **`/home/runwhen`**. + +--- + +## 5. Troubleshooting + +- Run **`gh codespace ssh -c YOUR_CODESPACE_NAME --`** again if the connection or auth state seems off. +- For **`authorized_keys`** permission errors inside the codespace: + + ```bash + sudo chmod 755 /home/runwhen + sudo chmod 700 /home/runwhen/.ssh + sudo chmod 600 /home/runwhen/.ssh/authorized_keys + ``` + +--- + +## 6. Codespace lifecycle + +Stop or delete codespaces in GitHub when you no longer need them. + +--- + +## 7. Local Docker and direct SSH (optional) + +The [`sshd` feature](https://github.com/devcontainers/features/tree/main/src/sshd) listens on port **2222** inside the container by default. This repo’s `devcontainer.json` forwards **3000** (logs) only; local workflows typically use **Dev Containers: Reopen in Container**. For SSH from the host, publish port **2222** and follow the feature docs (for example key-based login). + +--- + +## Related + +- `.devcontainer/devcontainer.json` +- [Repository README — Getting started](../README.md#getting-started) +- [sshd feature README](https://github.com/devcontainers/features/tree/main/src/sshd) diff --git a/docs/runwhen-keywords.md b/docs/runwhen-keywords.md deleted file mode 100644 index 4251909..0000000 --- a/docs/runwhen-keywords.md +++ /dev/null @@ -1,47 +0,0 @@ -# RunWhen Keywords - -### Running Bash and CLI Commands - -#### RW.Core.Create RunWhen Temp Dir - -* Most often Used in `Suite Setup`or `Suite Initialization`Tasks -* Creates a temporary directory and sets the Suite Variable `${RUNWHEN_TEMP_DIR}` with the value of the path -* Can easily be used to store/retrieve files or ouput for Task usage -* May automatically be cleaned up between Task runs - - - -Example Usage - -```robotframework -Suite Setup Suite Initialization -Suite Teardown Suite Teardown - -*** Keywords *** -Suite Initialization - RW.Core.Create RunWhen Temp Dir - - -*** Keywords *** -Run Kubectl CLI Command - [Documentation] Run Command and Store Output - [Tags] cli kubectl access:read-only - ${response}= RW.CLI.Run Cli - ... cmd=kubectl get pods -o json -A > ${RUNWHEN_TEMP_DIR}/pods.json - ... env=${env} - ... secret_file__kubeconfig=${kubeconfig} - -``` - -Related Keywords - -* `RW.Core.Create RunWhen Temp Dir` -* Often used in `Suite Teardown` - -Example Usage - -``` -Suite Teardown - RW.Core.Remove RunWhen Temp Dir - -```