Development

We welcome all contributors! Contributions to demes typically take the form of “pull requests” against our git repository.

Requirements

Demes aims to have minimal dependencies when used as a library by other projects. However, additional dependencies are required during development, as developers regularly run the test suite, build the documentation, and assess whether their code changes conform to style guidelines. The requirements.txt file in the top-level folder lists all development dependencies, which can be installed using pip. In the following documentation, we assume the reader has cloned the source repository and installed the developer requirements as follows.

# Clone the repository.
git clone https://github.com/popsim-consortium/demes-python.git
cd demes-python
# Create a virtual environment for development.
python -m venv venv
# Activate the environment.
source venv/bin/activate
# Install the developer dependencies.
pip install -r requirements.txt
# Generate the version string from the most recent git tag/commit.
python setup.py build

Warning

Due to conflicting version dependencies, it may not be possible to install all developer requirements on older versions of Python. If you experience problems, please install the latest Python version (3.10 at time of writing).

Note

Non-developer requirements are listed in the install_requires section of the setup.cfg file in the top-level folder of the sources.

Continuous integration (CI)

After a pull request is submitted, an automated process known as continuous integration (CI) will:

  • assess if the proposed changes conform to style guidelines (known as lint checks),

  • run the test suite,

  • and build the documentation.

The CI process uses GitHub Actions and the configuration files detailing how these are run can be found under the .github/workflows/ folder of the sources.

Lint checks

The following tools are run during the linting process:

  • black, a code formatter (code is only checked during CI, not reformatted),

  • flake8, a PEP8 code-style checker,

  • mypy, a static type checker.

Each of these tools can also be run manually from the top-level folder of the sources. The setup.cfg file includes some project-specific configuration for each of these tools, so running them from the command line should match the behaviour of the CI checks.

For example, to reformat the code with black after making changes to the demes/demes.py file (command output shown as comments):

black .
# reformatted /home/grg/src/demes/demes/demes.py
# All done! ✨ 🍰 ✨
# 1 file reformatted, 14 files left unchanged.

Similarly, one can check conformance to PEP8 style guidelines by running flake8 (without parameters), and check type annotations by running mypy (also without parameters).

Pre-commit

To simplify the process of running each of the linter tools, we also include a configuration for the pre-commit program. If you prefer not to run each of the commands above manually, install the pre-commit Python package and then run pre-commit install from the top-level folder of your cloned demes repository. Now when you try to commit changes, the lint checks will be performed automatically. If pre-commit made changes to the files when you tried to commit, you’ll need to stage those changes and try again. E.g. git add -u; git commit. See the pre-commit documentation for more details.

Test suite

A suite of tests is included in the tests/ folder. The CI process uses the pytest tool to run the tests, which can also be run manually from the top-level folder of the sources.

python -m pytest -v tests --cov=demes --cov-report=term-missing

This will produce lots of output, indicating which tests passed, and which failed (if any). There may also be warnings. Any warnings that are triggered by code in demes (rather than third-party libraries), should be fixed.

It is expected that new code contributions will be tested by the introduction of new tests in the test suite. While we don’t currently have any strict requirements for code coverage, more is better. Furthermore, we encourage contributions that improve, or expand on, the existing suite of tests.

Building the documentation

The demes documentation is built with jupyter-book, which uses sphinx. Much of the documentation is under the docs/ folder, written in the MyST flavour of Markdown, and is configured in the docs/_config.yml file. In contrast, the API documentation is automatically generated from “docstrings” in the Python code that use the reStructuredText format. To build the documentation locally, run make from the docs/ folder.

cd docs
make

If this was successful, the generated documentation can be viewed in a browser by navigating to the docs/_build/html/index.html file. It is expected that new code contributions will be accompanied by relevant documentation (e.g. a new function will include a docstring).

We strongly encourage contributions that improve the demes documentation, such as fixing typos and grammatical errors, or making the documentation clearer and/or more accessible.

Releasing a new version

  1. Prepare for the release by updating the CHANGELOG.md. The version here should have the format MAJOR.MINOR.PATCH, or for a beta release MAJOR.MINOR.PATCHbX, e.g. 1.0.0b1. Commit the changes to the changelog; make pull request; merge.

  2. Tag and push release. E.g. git tag 1.0.0b1 and git push origin 1.0.0b1. This triggers the wheel.yaml github workflow that uploads the release to https://test.pypi.org/project/demes/ and can be installed with: pip install demes --upgrade --pre --extra-index-url https://test.pypi.org/simple/.

  3. Create a release in the GitHub UI, based on the tag that was pushed (paste the output of docs/convert_changelog.py into the release notes). This will trigger the wheel.yaml workflow again, and upload the release to pypi.org. If you don’t wish to publish a beta release, this step can be omitted.

  4. Within a few hours, the conda-forge infrastructure will automagically detect the new release on PyPI and will make a pull request against the demes-feedstock repository to bump the version number. If there have been changes to dependencies, the conda recipe may need to be fixed by adding commits onto the bot’s pull request. Once satisfied, merge the pull request and the package will be conda-installable after a few minutes. See the conda-forge maintainer docs for additional information.