Testing Strategy

Testing Strategy#

This document outlines how io4dolfinx is tested, covering both local development testing and the Continuous Integration (CI) process on GitHub Actions.

Coverage reports:#

You will find the updated coverage reports for the latest version on main tested against stable and nightly versions of dolfinx at the following links:

Local Testing#

The library uses pytest for testing. To execute the tests locally, you first need to install the library and its dependencies.

Installation for Testing#

Install the library with the optional test dependencies to ensure you have packages like pytest, coverage, and ipyparallel

python3 -m pip install ".[test]"

Running Tests#

To execute all tests in the repository, run:

python3 -m pytest .

Generating Test Data#

Some tests require specific datasets to verify compatibility with older software versions.

Testing against data from legacy dolfin#

Some tests check the capability of reading data created with the legacy version of DOLFIN. To create this dataset, start a docker container with legacy DOLFIN, for instance:

docker run -ti -v $(pwd):/root/shared -w /root/shared --rm ghcr.io/scientificcomputing/fenics:2024-02-19

Then, inside this container, call:

python3 ./tests/create_legacy_data.py --output-dir=legacy

Testing against data from older versions of adios4dolfinx#

Some tests check the capability to read data generated by adios4dolfinx<0.7.2. To generate data for these tests use the following commands:

docker run -ti -v $(pwd):/root/shared -w /root/shared --rm ghcr.io/fenics/dolfinx/dolfinx:v0.7.3

Then, inside the container, call:

python3 -m pip install adios4dolfinx==0.7.1
python3 ./tests/create_legacy_checkpoint.py --output-dir=legacy_checkpoint

Continuous Integration (GitHub Actions)#

The repository relies on several GitHub Actions workflows to ensure code quality and compatibility across different environments.

1. Main Test Suite (test_package.yml)#

This is the primary workflow triggered on pushes to main, pull requests, and scheduled nightly runs. It runs on ubuntu-24.04 using the official DOLFINx docker container .

Workflow Steps:

  1. Linting & Formatting: Checks code style using ruff and type consistency with mypy .

  2. Data Generation: * Creates legacy DOLFIN data using the create_legacy_data.yml workflow.

    • Creates legacy adios4dolfinx checkpoints using create_legacy_checkpoint.yml.

  3. Test Execution:

    • Installs the package with MPI-enabled h5py.

    • Runs the standard test suite with coverage.

    • Runs parallel tests using mpirun -n 4 ... mpi4py -m pytest.

  4. Reporting: Combines coverage reports and uploads them as artifacts.

2. Compatibility Testing#

To ensure broad support, specific workflows test against different configurations:

  • MPI & ADIOS2 Versions (test_package_openmpi.yml):

    • Tests against both openmpi and mpich implementations using the ghcr.io/fenics/test-env containers.

    • Verifies compatibility with different ADIOS2 versions (e.g., v2.10.2, v2.11.0) .

  • Operating System (test_redhat.yml):

    • Runs the full test suite inside a RedHat-based container (docker.io/fenicsproject/test-env:current-redhat) to guarantee functionality on non-Debian systems .

3. Documentation (build_docs.yml)#

Ensures the documentation builds correctly with jupyter-book on every push and pull request, preventing documentation regressions.

Contents