From d9a42968a27f2587ffb1e015ce14c014dd9f786e Mon Sep 17 00:00:00 2001 From: James Date: Wed, 7 Dec 2022 17:19:02 +0000 Subject: [PATCH 1/5] docs/standard/contributing: Add Github Actions tests notes --- docs/standard/contributing.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/standard/contributing.md b/docs/standard/contributing.md index 84488f67..7ead903b 100644 --- a/docs/standard/contributing.md +++ b/docs/standard/contributing.md @@ -101,6 +101,36 @@ To improve your technical writing skills, consider taking [Google's Technical Wr 1. Once approved, you can merge it yourself. +## Pull Request Tests + +When you submit a pull request, some tests will be run against it automatically. + +### Linkcheck + +This will check all external links are valid, and error if not. + +If this fails for your pull request, that is ok. This may be due to existing problems or a link in another section of the docs that used to work but now doesn't. + +Check the details to make sure that any failures aren't due to any links you have added, and if so attempt to fix them. + +### Tests + +Various Python tests will run. This must pass before your pull request can be merged. + +To run this locally, see [instructions on the build page](../technical/index). + +### Mdformat + +This formats the md files to a standard specification. This must pass before your pull request can be merged. + +This should be done by your pre-commit hook. If this fails, make sure your pre-commit hook is running correctly. + +### Build test + +This tries to build the site, and treats any warnings as problems as well as errors. This must pass before your pull request can be merged. + +To run this locally, see [instructions on the build page](../technical/index). + ## Logging changes 1. Follow the [changelog style guide](../style/changelog_style_guide). From 5e77eb330817d5ccd43043f46ca0564475abcb3c Mon Sep 17 00:00:00 2001 From: Duncan Dewhurst Date: Fri, 16 Dec 2022 15:15:39 +1300 Subject: [PATCH 2/5] Refactor checks and tests documentation --- docs/standard/contributing.md | 32 +-------------- docs/technical/build.md | 2 +- docs/technical/checks.md | 76 +++++++++++++++++++++++++++++++++++ docs/technical/index.md | 19 +-------- 4 files changed, 80 insertions(+), 49 deletions(-) create mode 100644 docs/technical/checks.md diff --git a/docs/standard/contributing.md b/docs/standard/contributing.md index 7ead903b..e28300b2 100644 --- a/docs/standard/contributing.md +++ b/docs/standard/contributing.md @@ -94,6 +94,8 @@ To improve your technical writing skills, consider taking [Google's Technical Wr - Set the *base* branch, e.g. `main`. - Set the *milestone*, e.g. `1.0`. +1. Resolve any failing [checks](../technical/checks.md) + 1. Assign to another team member to review. - See the next section for reviewer's instructions. @@ -101,36 +103,6 @@ To improve your technical writing skills, consider taking [Google's Technical Wr 1. Once approved, you can merge it yourself. -## Pull Request Tests - -When you submit a pull request, some tests will be run against it automatically. - -### Linkcheck - -This will check all external links are valid, and error if not. - -If this fails for your pull request, that is ok. This may be due to existing problems or a link in another section of the docs that used to work but now doesn't. - -Check the details to make sure that any failures aren't due to any links you have added, and if so attempt to fix them. - -### Tests - -Various Python tests will run. This must pass before your pull request can be merged. - -To run this locally, see [instructions on the build page](../technical/index). - -### Mdformat - -This formats the md files to a standard specification. This must pass before your pull request can be merged. - -This should be done by your pre-commit hook. If this fails, make sure your pre-commit hook is running correctly. - -### Build test - -This tries to build the site, and treats any warnings as problems as well as errors. This must pass before your pull request can be merged. - -To run this locally, see [instructions on the build page](../technical/index). - ## Logging changes 1. Follow the [changelog style guide](../style/changelog_style_guide). diff --git a/docs/technical/build.md b/docs/technical/build.md index c0b449bb..3f4b3958 100644 --- a/docs/technical/build.md +++ b/docs/technical/build.md @@ -57,4 +57,4 @@ Versions are managed via [readthedocs](https://readthedocs.org/) admin (credenti ``` https://open-fibre-data-standard.readthedocs.io/en/my-development-branch-name/ -``` \ No newline at end of file +``` diff --git a/docs/technical/checks.md b/docs/technical/checks.md new file mode 100644 index 00000000..d4352288 --- /dev/null +++ b/docs/technical/checks.md @@ -0,0 +1,76 @@ +# Checks and tests + +The standard repository uses [GitHub Actions](https://docs.github.com/en/actions) to run automated checks each time you push a commit to GitHub. + +With the exception of [Link Check](#link-check), all checks must pass successfully before you can merge a pull request. + +Checks are defined by the YAML files in the [`.github/workflows` directory](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/tree/0.1-dev/.github/workflows). + +This page describes the purpose of each check, how to run it locally and what to do about failures. + +Before running checks locally, you need to follow the [getting started instructions](build.md#get-started). + +## Link Check + +Check the integrity of all external links. + +To run locally: + +```bash +cd docs +make linkcheck +``` + +To correct failures, review the output and fix broken links, where possible. This check may report false positives, for example, when linking to anchors in Markdown files hosted on GitHub. Therefore, failures do not block merging. + +## MD Format + +Enforce a consistent style in Markdown files. + +To run locally: + +```bash +mdformat --check docs +``` + +To correct failures, run the following command: + +```bash +./manage.py pre-commit +``` + +## Sphinx Build + +Build the documentation and warn about all references where the target cannot be found. + +To run locally: + +```bash +sphinx-build -nW ./docs/ ./temp_build +``` + +To correct failures, review the output and fix broken references. + +## Run all tests + +Run the tests defined in the [`/tests` directory](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/tree/0.1-dev/tests). + +Tests are written using the [pytest framework](https://docs.pytest.org/en/7.2.x/) and use methods from [JSON Schema and CSV Codelists](https://jscc.readthedocs.io/en/latest/). + +There are two test files: + +* `test_csv.py` - Ensure that all CSV files are valid: no empty rows or columns, no leading or trailing whitespace in cells, same number of cells in each row. +* `test_json.py`- Ensure that: + * All JSON files are non-empty, correctly indented and valid. + * Codes in codelist CSV files match enums in the schema. + * Codelist CSV files are referenced in the schema. + * Codelists CSV files referenced in the schema exist. + * JSON Schema files are valid. + +To run locally: + +```bash +pytest tests +``` + +To correct failures, review the warnings and recommended actions in the output. diff --git a/docs/technical/index.md b/docs/technical/index.md index a6d89781..bd89a7bb 100644 --- a/docs/technical/index.md +++ b/docs/technical/index.md @@ -5,23 +5,6 @@ :glob: build +checks update ``` - -## Running tests locally - -When you push code to GitHub, tests are automatically run. You will not be allowed to merge your pull request until these tests pass. - -To run these locally, first create and install a virtual environment [as instructed on the build page](build). - -To run tests locally, run: - -``` -pytest tests/ -``` - -To build the project while checking for warnings, run: - -``` -sphinx-build -nW ./docs/ ./temp_build -``` From c7966a1199294f7d7462aaf57936855e6ef1b13e Mon Sep 17 00:00:00 2001 From: Duncan Dewhurst Date: Fri, 16 Dec 2022 15:18:38 +1300 Subject: [PATCH 3/5] docs/standard/contributing.md: Remove repeated content from PR template --- docs/standard/contributing.md | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/docs/standard/contributing.md b/docs/standard/contributing.md index e28300b2..d95ae121 100644 --- a/docs/standard/contributing.md +++ b/docs/standard/contributing.md @@ -28,18 +28,7 @@ To improve your technical writing skills, consider taking [Google's Technical Wr - **Never** use normative words on guidance pages. Use [non-normative synonyms](https://tools.ietf.org/html/draft-hansen-nonkeywords-non2119-04#page-3) instead. - Consider composing Markdown content in [Hemingway Editor](http://www.hemingwayapp.com/) or [Grammarly](https://www.grammarly.com/), as suggested in the [common conventions](../style/common_conventions.md). - - After adding a definition to `schema.json`, add a sub-heading for the new sub-schema in `reference.md`. - -1. Update the **changelog**, maintaining schema order in the list of changes. -1. Run `./manage.py pre-commit` to update the reference documentation -1. If you used a validation keyword, type or format that is not [already used in the schema](schema.md#json-schema-usage): - 1. Update the list of validation keywords, types or formats in [JSON Schema usage](schema.md#json-schema-usage). - 1. Add a field that fails validation against the new keyword, type or format to [`network-package-invalid.json`](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/blob/0.1-dev/examples/json/network-package-invalid.json). - 1. Check that [OFDS CoVE](https://ofds.cove.opendataservices.coop/) reports an appropriate validation error. -1. If you added a normative rule that is not encoded in JSON Schema: - 1. Update the list of [other normative rules](schema.md#other-normative-rules). - 1. Add a field that does not conform to the rule to [`network-package-additional-checks.json`](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/blob/0.1-dev/examples/json/network-package-additional-checks.json)/ - 1. Open a [new issue](https://github.com/Open-Telecoms-Data/lib-cove-ofds/issues/new/choose) to add an additional check to Lib Cove OFDS. + 1. Commit your changes. ```{admonition} Atomic changes @@ -90,10 +79,12 @@ To improve your technical writing skills, consider taking [Google's Technical Wr 1. Create a pull request. - - Write the pull requests' description, filling in [the relevant bits of the provided template](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/blob/0.1-dev/pull_request_template.md). + - Complete the pull request template. - Set the *base* branch, e.g. `main`. - Set the *milestone*, e.g. `1.0`. +1. Complete the merge checklist. + 1. Resolve any failing [checks](../technical/checks.md) 1. Assign to another team member to review. From 4bd193765956e539fb2cbd7c40cd288cbb2fa1c8 Mon Sep 17 00:00:00 2001 From: Duncan Dewhurst Date: Fri, 16 Dec 2022 15:36:38 +1300 Subject: [PATCH 4/5] Move Read the Docs documentation to deployment page --- docs/technical/build.md | 10 ---------- docs/technical/deployment.md | 20 ++++++++++++++++++++ docs/technical/index.md | 1 + 3 files changed, 21 insertions(+), 10 deletions(-) create mode 100644 docs/technical/deployment.md diff --git a/docs/technical/build.md b/docs/technical/build.md index 3f4b3958..b09c474e 100644 --- a/docs/technical/build.md +++ b/docs/technical/build.md @@ -34,7 +34,6 @@ cd docs make dirhtml ``` - Sphinx, which builds the documentation, doesn’t watch directories for changes. To regenerate the documentation and refresh the browser whenever changes are made, run: ``` @@ -49,12 +48,3 @@ python -m http.server ``` Then go to http://localhost:8000/ in a browser. - -## View development documentation on readthedocs - -Versions are managed via [readthedocs](https://readthedocs.org/) admin (credentials needed). Automation rules have been set up to automatically build and activate hidden versions when new branches are created. Therefore development branches will not be available from the flyout menu on readthedocs, but can be viewed by inserting the branch name into the url, as follows: - -``` -https://open-fibre-data-standard.readthedocs.io/en/my-development-branch-name/ - -``` diff --git a/docs/technical/deployment.md b/docs/technical/deployment.md new file mode 100644 index 00000000..eebd82c7 --- /dev/null +++ b/docs/technical/deployment.md @@ -0,0 +1,20 @@ +# Deployment + +[Read the Docs](https://readthedocs.org/) is used to build and deploy the documentation. + +[Automation rules](https://docs.readthedocs.io/en/stable/automation-rules.html) are used to: + +* Build a version of the documentation for each branch in the standard repository +* Delete a version when a branch is deleted + +Each time you push a commit to a branch, Read the Docs builds the version for that branch. In addition to the versions for each branch, there is also: + +* A `latest` version that redirects to the `0.1` branch. +* A `stable` version that redirects to the most recent release. +* A version for each [tag](https://github.com/Open-Telecoms-Data/open-fibre-data-standard/tags) in the standard repository, e.g. `0__1__0__beta`. + +Only the `latest` version is available in the [flyout menu](https://docs.readthedocs.io/en/stable/glossary.html#term-flyout-menu). To view other versions, substitute the branch name into the URL, as follows: + +``` +https://open-fibre-data-standard.readthedocs.io/en/branch-name/ +``` diff --git a/docs/technical/index.md b/docs/technical/index.md index bd89a7bb..450ceb6d 100644 --- a/docs/technical/index.md +++ b/docs/technical/index.md @@ -6,5 +6,6 @@ build checks +deployment update ``` From 7c708ebc70a3c4b05ae5640c9f705345821933fe Mon Sep 17 00:00:00 2001 From: Duncan Dewhurst Date: Fri, 16 Dec 2022 15:38:48 +1300 Subject: [PATCH 5/5] docs/standard/contributing.md: copy-edit --- docs/standard/contributing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/standard/contributing.md b/docs/standard/contributing.md index d95ae121..9cc9148b 100644 --- a/docs/standard/contributing.md +++ b/docs/standard/contributing.md @@ -79,9 +79,9 @@ To improve your technical writing skills, consider taking [Google's Technical Wr 1. Create a pull request. - - Complete the pull request template. - - Set the *base* branch, e.g. `main`. - - Set the *milestone*, e.g. `1.0`. + - Complete the 'Related issues' and 'Description' sections of the pull request template. + - Set the *base* branch, e.g. `0.1-dev`. + - Set the *milestone*, e.g. `Beta`. 1. Complete the merge checklist.