Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/publish-llm-markdown.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ jobs:
rsync -av --exclude '_build' --exclude 'templates' _source/validmind-library/docs/ validmind/
working-directory: site

- name: Install pandoc
run: |
sudo apt-get update
sudo apt-get install -y pandoc

- name: Render LLM markdown
run: bash llm/render.sh
working-directory: site
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/validate-docs-site.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,11 @@ jobs:
body: comment
});

- name: Install pandoc
run: |
sudo apt-get update
sudo apt-get install -y pandoc

- name: Validate LLM markdown render
run: bash llm/render.sh && bash llm/clean.sh
working-directory: site
Expand Down
9 changes: 8 additions & 1 deletion site/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ SRC_ROOT := _source
SRC_DIR := $(SRC_ROOT)/validmind-library

# Define .PHONY target for help section
.PHONY: help add-copyright clean clone copy-installation copy-release-notes delete-demo-branch deploy-demo-branch deploy-prod deploy-staging docker-build docker-serve docker-site docker-site-lite docs-site execute generate-sitemap get-api-json get-source kind-serve kind-stop kind-restart kind-logs notebooks python-docs release-notes test-descriptions verify-copyright yearly-releases
.PHONY: help add-copyright clean clone copy-installation copy-release-notes delete-demo-branch deploy-demo-branch deploy-prod deploy-staging docker-build docker-serve docker-site docker-site-lite docs-site execute generate-sitemap get-api-json get-source kind-serve kind-stop kind-restart kind-logs notebooks python-docs release-notes render-llm test-descriptions verify-copyright yearly-releases

# Help section
help:
Expand Down Expand Up @@ -49,6 +49,7 @@ help:
@echo " python-docs Copy the Python library docs into validmind/"
@echo " release-notes TAG=<tag> Clone and copy release notes for the specified tag or repo/tag"
@echo " Examples: TAG=cmvm/25.07 or TAG=validmind-library/v2.8.22"
@echo " render-llm Render site to GFM markdown for LLM ingestion (mirrors CI)"
@echo " test-descriptions Copy the ValidMind tests docs into tests/"
@echo " verify-copyright Verify that all .qmd and .yml/.yaml files have copyright headers"
@echo " yearly-releases Collate releases by year into a listing landing and update releases sidebar"
Expand Down Expand Up @@ -434,3 +435,9 @@ yearly-releases:
cd ../
git status | grep -v 'release-scripts/'
quarto preview

# Render site to GFM markdown for LLM ingestion
render-llm:
@echo "\nRendering site to GFM markdown for LLM ingestion ..."
@bash llm/render.sh && bash llm/clean.sh
@echo "\nOutput in llm/_llm-output/"
55 changes: 2 additions & 53 deletions site/developer/how-to/testing-overview.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -80,33 +80,7 @@ listing:

## Running a test

```{dot}
digraph {
rankdir=LR
bgcolor="transparent"
margin=0
pad=0.1
ranksep=0.5
nodesep=0.8
node [shape=box, style="rounded,filled", fillcolor="#FFFFFF", color="#083E44", fontcolor="#36454F", fontname="Arial,Helvetica,sans-serif", fontsize=10, penwidth=0.7, margin="0.24,0.3"]
edge [color="#083E44", penwidth=0.7]
graph [fontcolor="#36454F", fontname="Arial,Helvetica,sans-serif", fontsize=10]

subgraph cluster_identify { label="Identify"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; TestID [label="Test ID"]; ResultID [label="Result ID"] }
subgraph cluster_configure { label="Configure"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; Inputs; Params [label="Parameters"] }
subgraph cluster_run { label="Run"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; RunTest [label="run_test()"] }
subgraph cluster_output { label="Outputs & log results"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; Output [label="Tables, plots, etc."]; Log [label=".log()"] }
Platform

TestID -> RunTest
ResultID -> RunTest [style=dashed]
Inputs -> RunTest
Params -> RunTest
RunTest -> Output
Output -> Log
Log -> Platform
}
```
![](testing-overview_files/figure-commonmark/dot-figure-1.png)

**Test ID**
: Each test has a unique identifier like `validmind.data_validation.ClassImbalance`. Use `vm.tests.list_tests()` to browse available tests.
Expand Down Expand Up @@ -138,32 +112,7 @@ Use `vm.tests.describe_test("test_id")` or check the [test descriptions](/develo

## Running a test suite

```{dot}
digraph {
rankdir=LR
bgcolor="transparent"
margin=0
pad=0.1
ranksep=0.5
nodesep=0.8
node [shape=box, style="rounded,filled", fillcolor="#FFFFFF", color="#083E44", fontcolor="#36454F", fontname="Arial,Helvetica,sans-serif", fontsize=10, penwidth=0.7, margin="0.24,0.3"]
edge [color="#083E44", penwidth=0.7]
graph [fontcolor="#36454F", fontname="Arial,Helvetica,sans-serif", fontsize=10]

subgraph cluster_identify { label="Identify"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; SuiteID [label="Test suite ID"] }
subgraph cluster_configure { label="Configure"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; SuiteInputs [label="Inputs"]; SuiteConfig [label="Config"] }
subgraph cluster_run { label="Run"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; RunSuite [label="run_test_suite()"] }
subgraph cluster_output { label="Outputs & log results"; style="rounded"; color="#083E44"; fillcolor="#FFFFFF"; margin=6; SuiteOutput [label="Tables, plots, etc."]; SuiteLog [label=".log()"] }
Platform

SuiteID -> RunSuite
SuiteInputs -> RunSuite
SuiteConfig -> RunSuite
RunSuite -> SuiteOutput
SuiteOutput -> SuiteLog
SuiteLog -> Platform
}
```
![](testing-overview_files/figure-commonmark/dot-figure-2.png)

**Test suite ID**
: Test suites are collections of tests that run together. Each has an identifier like `classifier_full_suite`. Use `vm.tests.list_test_suites()` to browse available test suites.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading