KEH Policy Methods Library

The Policy Methods Library is a collection of functions to encapsulate the business logic when it comes checking policy adherence to the GitHub Usage Policy within ONS.

Table of Contents

• KEH Policy Methods Library
  • Table of Contents
  • Prerequisites
  • Makefile
  • Using the Package
    • End User Instructions
    • Developer Instructions
  • Package Structure
  • Deployment
  • Documentation
    • GitHub Actions for Documentation
    • Local Development of Documentation
  • Linting and Testing
    • GitHub Actions
    • Running Tests and Linters Locally
      • Primary Language
      • MegaLinter
      • Documentation linting and building

Prerequisites

• Python 3.12 or higher
• Poetry for dependency management
• Node.js and npm for documentation linting (Markdownlint)

Makefile

This project uses a Makefile to simplify common tasks. To see the available commands, run:

make help

Using the Package

End User Instructions

To use the package, you can install it using your preferred method (e.g. pip, Poetry, etc.) and then import the relevant modules and functions in your code.

# Using pip
pip install git+https://github.com/ONS-Innovation/keh-policy-methods-library@<version>

# or using Poetry
poetry add git+https://github.com/ONS-Innovation/keh-policy-methods-library@<version>

Then, in your Python code, you can import the relevant modules and functions:

from policy_methods_library.github.clients import GitHubRestClient

client = GitHubRestClient(
    owner=github_organisation,
    app_id=app_id,
    private_key=private_key,
)

Detailed examples of the functionality provided by the package can be found in the documentation, which is available in the docs directory and deployed to GitHub Pages.

Developer Instructions

When developing the package, you can use the tests/manual_testing.py file for manual testing of the package during development. This file has a GitHub Client instance already setup which can be used to test the checks in the policy_methods_library.

To use the latest version of the package in the manual_testing.py file, you can install the package into your virtual environment:

pip install .

Note: This should be done from the root directory of the repository, and you should have your virtual environment activated.

Whenever you make changes to the package, you will need to reinstall it for the changes to be reflected in the manual_testing.py file.

Further instructions for developers, including how to add new checks, can be found in the documentation under the "Dev Guides" section.

Package Structure

src/
└── policy_methods_library/
   ├── checks/                # Core business logic for checking policy adherence.
   │   ├── __init__.py
   │   └── ...                # files for each specific check
   │
   ├── github/                # GitHub API integrations.
   │   ├── __init__.py
   │   ├── auth.py            # Functions to handle authentication with the GitHub API.
   │   └── clients.py         # Client for making REST API calls to GitHub.
   │
   └── __init__.py            # Init file for the package.

Further documentation can be found within the docs directory.

Deployment

The package can be deployed to GitHub Releases using the GitHub Actions workflow defined in .github/workflows/publish-release.yml.

This workflow is triggered on pushes to tags matching the pattern "v*". The tag must follow the format "v0.0.0" (e.g. "v0.1.0", "v1.0.0", etc.) for the workflow to run successfully.

A production release will be created when a tag is pushed that follows the format "v0.0.0" (e.g. "v0.1.0", "v1.0.0", etc.). A pre-release will be created when a tag is pushed that follows the format "v0.0.0-rc" (e.g. "v0.1.0-rc", "v1.0.0-rc", etc.). This allows for both stable releases and pre-releases to be created and published to GitHub Releases.

To create a new release, you can use the following command to create a new tag and push it to the repository:

git tag v0.1.0
git push origin v0.1.0

This will tag the current commit with the version "v0.1.0" and push the tag to the remote repository, which will trigger the GitHub Actions workflow to build and publish the package to GitHub Releases.

Note: Is is important that GitHub Releases are only created via the GitHub Actions workflow, and not manually via the GitHub UI. This is because the workflow ensures that the package is built and published correctly.

Documentation

This repository uses MkDocs for documentation. The documentation source files are located in the docs directory.

GitHub Actions for Documentation

MkDocs gets deployed to GitHub Pages using GitHub Actions. The workflow for this is located at .github/workflows/deploy-docs.yml. Before deployment, another GitHub Action workflow runs to check that the documentation builds correctly and has no linting or formatting issues. This workflow is located at .github/workflows/ci-docs.yml.

Local Development of Documentation

To run the documentation locally:

1. Create a Python virtual environment and activate it.

   python -m venv venv
   source venv/bin/activate

2. Install the dependencies for MkDocs.

   make docs-install

3. Run the MkDocs development server.

   make docs-serve

Linting and Testing

GitHub Actions

This repository has GitHub Actions workflows set up for linting and testing. The workflows are located at:

• .github/workflows/ci-fmt.yml for linting and formatting checks (primary language).
• .github/workflows/ci-test.yml for running automated tests.
• .github/workflows/ci-docs.yml for checking that the documentation builds correctly and has no linting or formatting issues.
• .github/workflows/megalinter.yml for running MegaLinter, which checks for linting and formatting issues across multiple languages and file types (this is a catch-all linter).
• .github/workflows/deploy-docs.yml for deploying documentation to GitHub Pages.

Running Tests and Linters Locally

Primary Language

To run the linters and formatters for the primary language (Python) locally, you can use the following command:

make lint

To apply automatic fixes for any linting or formatting issues found, you can use:

make fmt

To run the tests locally, you can use:

make test-unit

MegaLinter

This repository uses MegaLinter for comprehensive linting across multiple languages and file types. We use this so that all additional assets in the repository (e.g. YAML files, Markdown files, etc.) are also linted and checked for formatting issues, without having to set up specific linters for each file type.

To run MegaLinter locally, you can use the following command:

make megalinter

Documentation linting and building

This repository uses Markdownlint for linting the documentation. To run Markdownlint locally, you can use the following:

make docs-lint

Note: This will install markdownlint-cli globally via npm if it is not already installed.

To apply automatic fixes for any linting issues found by Markdownlint, you can use:

make docs-fix

To test that the documentation builds correctly, you can use the following command:

make docs-build

Note: This depends on MkDocs being set up for the repository. Instructions for setting up MkDocs can be found in the Documentation section of this README.