Skip to content

Configuration and automation

Authoritative checks

Run the complete local validation suite with:

make check

It verifies the lockfile, Ruff, Markdown, Dockerfiles, GitHub Actions workflows, repository workflow environment references, spelling, secret scanning, Bandit, deptry, pip-audit, tests and coverage, minimum direct dependency versions, basedpyright, documentation links, distribution metadata, and SBOM generation.

Dependency groups

Python development dependencies are organized into focused dependency groups: test, lint, type, docs, security, release, and automation. The aggregate dev group includes all of them and remains the default local development environment used by make install, CI, nox, and the devcontainer.

Use focused groups only when you intentionally need a smaller environment, for example:

uv sync --locked --only-group docs
uv sync --locked --only-group security

Documentation

Build the documentation site with:

make docs

Check links in the built site with:

make docs-linkcheck

make docs-linkcheck runs mkdocs build --strict and then checks the generated site with LinkChecker. API pages under reference/api/ are generated from package docstrings during the build.

MkDocs is configured to treat documentation warnings as failures. Navigation entries, local links, anchors, and generated API reference pages must stay in sync with the source tree. The local development server also watches src/benchmatrix/ so docstring changes can refresh generated reference pages.

Spelling checks include regular repository files and hidden GitHub configuration under .github/, so issue templates, workflow names, and pull request template text stay covered by the same make spellcheck target.

Repository settings

Repository settings that can be managed as code are declared in .github/settings.yml. The file is intended for the GitHub Settings app and covers repository metadata, merge strategy defaults, vulnerability alert settings, and main branch protection. External setup that still requires a service UI is tracked in the external repository setup runbook.

GitHub Actions workflows

The repository uses focused workflows that call the same Make targets used locally:

  • .github/workflows/ci.yml runs quality checks, supported-Python tests, coverage and JUnit report artifact upload, minimum-dependency tests, packaging smoke checks, SBOM generation, and scheduled dependency audits.
  • .github/workflows/docs.yml builds docs, checks generated-site links, uploads the link-check report, and deploys the MkDocs site from main through GitHub Pages.
  • .github/workflows/draft-release.yml validates release metadata when a v* tag is pushed, extracts release notes from CHANGELOG.md, builds and verifies release assets, and creates or updates the draft GitHub Release with the package source distribution, wheel, and SBOM attached; it also supports manual dispatch for retrying an existing tag.
  • .github/workflows/release.yml builds release artifacts, uploads package distributions and the SBOM as separate Actions artifacts, attaches them to the GitHub Release, attests them, publishes the distributions to PyPI through Trusted Publishing, and verifies installation from PyPI after publication.
  • .github/workflows/release-verify.yml manually re-runs post-release installation verification from PyPI.
  • .github/workflows/docker.yml builds runtime and test Docker images, scans them for critical vulnerabilities, and publishes them to GHCR from main and v* tags.
  • .github/workflows/workflow-lint.yml runs actionlint and zizmor for workflow configuration changes, using .github/zizmor.yml for project-specific audit policy.
  • .github/workflows/labeler.yml applies pull request labels from .github/labeler.yml without checking out untrusted pull request code.
  • .github/workflows/codeql.yml, .github/workflows/dependency-review.yml, and .github/workflows/scorecard.yml provide GitHub-native security and supply-chain checks.

Standalone SBOM and artifact-attestation workflows are intentionally not added: make build, CI packaging, and the release workflow already generate SBOMs and release provenance without an extra scheduler. Release drafting is limited to a small tag-triggered workflow so the deployment action remains publishing the reviewed GitHub Release.

Pull request labels

Pull request labels are applied automatically by GitHub Actions:

  • .github/labeler.yml maps changed files and branch names to labels.
  • .github/workflows/labeler.yml runs actions/labeler on pull requests.
  • .github/labels.yml documents the repository's recommended label set for maintainers.

Labels are not synced automatically. Create or update labels manually in GitHub when the recommended label set changes.

Dependabot

Dependency version update automation is configured in .github/dependabot.yml. It runs weekly, uses conservative pull request limits, and covers the ecosystems that have manifests in this repository:

  • uv for pyproject.toml and uv.lock;
  • npm for Markdown and spelling tooling in package.json;
  • pre-commit for hook revisions in .pre-commit-config.yaml;
  • github-actions for workflow actions under .github/workflows/;
  • docker for the runtime Docker image and devcontainer base images;
  • devcontainers for devcontainer features.

Dependabot pull requests should be labeled with dependencies and maintenance, plus a more specific label such as github-actions, automation, or dev-environment where applicable.

Multi-version automation

Run tests against the minimum direct dependency set on the oldest supported Python version with:

make test-min-deps

Run the nox matrix with:

make test-matrix

Useful focused sessions:

uv run nox -s tests-3.11
uv run nox --tags quality
uv run nox --tags docs
uv run nox -s release

Docker images

Build and smoke-test the runtime image locally with:

make docker-build
make docker-smoke

Build and run the test image locally with:

make docker-test

Run all local Docker checks with:

make docker-check

Scan locally built images for critical vulnerabilities with:

make docker-scan

make check runs Dockerfile linting but does not build Docker images, so normal local validation does not require a Docker daemon. The devcontainer includes the Docker-outside-of-Docker feature so contributors can run make docker-check after rebuilding the devcontainer. The Docker workflow builds, scans, and publishes images in GitHub Actions.

Generated artifacts

  • dist/ contains built distributions and generated SBOMs.
  • reports/ contains local test, coverage, and link-check reports.
  • site/ contains the local MkDocs output.
  • .nox/ contains local nox environments.

These directories are local artifacts and should not be committed.