Skip to content

Commit

Permalink
minor tweaks, some improvement of dev guide
Browse files Browse the repository at this point in the history
  • Loading branch information
a.pirogov committed Jul 24, 2023
1 parent c2b8c20 commit ea16806
Show file tree
Hide file tree
Showing 9 changed files with 141 additions and 78 deletions.
2 changes: 1 addition & 1 deletion .somesy.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "fair-python-cookiecutter"
version = "0.1.1"
version = "0.1.0"
description = "An opinionated cookiecutter template to kickstart a modern best-practice Python project with FAIR metadata. "

license = "MIT"
Expand Down
7 changes: 1 addition & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,7 @@ Here we provide notes that summarize the most important changes in each released

Please consult the changelog to inform yourself about breaking changes and security issues.

## [v0.1.1](https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter/tree/v0.1.0) <small>(2023-05-11)</small> { id="0.1.1" }

* Add [somesy](https:///Materials-Data-Science-and-Informatics/somesy) to manage and synchronize project metadata
* Various minor fixes

## [v0.1.0](https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter/tree/v0.1.0) <small>(2023-05-11)</small> { id="0.1.0" }
## [v0.1.0](https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter/tree/v0.1.0) <small>(2023-07-24)</small> { id="0.1.0" }

* First release

2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ type: software
message: If you use this software, please cite it using this metadata.

title: "fair-python-cookiecutter"
version: "0.1.1"
version: "0.1.0"
abstract: "An opinionated cookiecutter template to kickstart a modern best-practice
Python project with FAIR metadata."
repository-code: "https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter"
Expand Down
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,40 @@ default_context:
This information will be already pre-filled when you use the template,
saving you some time and possibly avoiding possible mistakes from manual typing.
## Modifying the Template
If you want to adjust it to your needs and likings (e.g. add, remove or substitute certain
tools), you probably want to
[fork](https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter/fork)
it to get your own copy. Then you can do the desired changes and use the URL of your
template repository instead of this one to kickstart your projects.
However, if you think that your changes are of general interest and would improve this
template, consider to get in touch
and [contribute](https://materials-data-science-and-informatics.github.io/fair-python-cookiecutter/main/contributing/)!
In any case we are very happy to know about any similar or derivative templates, e.g. for
more specific use-cases or based on other tool preferences.
## Reusing Parts of the Template
If you already have an existing project where you would like to introduce things you like
from this template, there are two main ways to do so:
1. move your code into a fresh repository based on this template
2. use parts of the template in your existing project structure
If your project currently has no sophisticated setup of tools or strong preferences about
them, option 1 might be the simplest way to adopt the template. Your code then needs to be
moved into the `YOUR_PROJECT/src` subdirectory.

On the other hand, if you already have a working setup that you do not wish to replace
completely, you can take a look at

* the `.pre-commit-config.yaml` file to adopt some of the quality assurance tools listed there
* the CI pipelines defined in `.github/workflows` or `.gitlab-ci.yml` for automated tests and releases
* the `mkdocs.yml` and `docs/` subdirectory to see how the project website works

<!-- --8<-- [end:quickstart] -->

<!-- --8<-- [start:citation] -->
Expand Down
6 changes: 3 additions & 3 deletions codemeta.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
"audience": [
{
"@type": "Audience",
"audienceType": "Developers"
"audienceType": "Science/Research"
},
{
"@type": "Audience",
"audienceType": "Science/Research"
"audienceType": "Developers"
}
],
"author": [
Expand Down Expand Up @@ -59,5 +59,5 @@
}
],
"url": "https://materials-data-science-and-informatics.github.io/fair-python-cookiecutter",
"version": "0.1.1"
"version": "0.1.0"
}
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ repo_url: "https://github.com/Materials-Data-Science-and-Informatics/fair-python
edit_uri: "blob/main/docs/"
repo_name: "Materials-Data-Science-and-Informatics/fair-python-cookiecutter"
docs_dir: "{{ cookiecutter.__project_slug }}/docs"
watch: ["README.md"]

# navigation structure (TOC + respective markdown files):
nav:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[tool.poetry]
# ---- managed by somesy, see .somesy.toml ----
name = "fair-python-cookiecutter"
version = "0.1.1"
version = "0.1.0"
description = "An opinionated cookiecutter template to kickstart a modern best-practice Python project with FAIR metadata."
authors = ["Anton Pirogov <a.pirogov@fz-juelich.de>"]
maintainers = ["Anton Pirogov <a.pirogov@fz-juelich.de>"]
Expand Down
163 changes: 98 additions & 65 deletions {{ cookiecutter.__project_slug }}/docs/dev_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,66 +9,75 @@ and provides more information on how to work with this repository.

Here is a *non-exhaustive* list of the most important files and directories in the repository.

*General:*

* `AUTHORS.md`: acknowledges and lists all contributors
* `CHANGELOG.md`: summarizes the changes for each version of the software for users
* `CODE_OF_CONDUCT.md`: defines the social standards that must be followed by contributors
* `CONTRIBUTING.md`: explains how others can contribute to the project
* `README.md`: provides an overview and points to other resources

*Metadata:*

* `CITATION.cff`: metadata stating how to cite the project
* `codemeta.json`: metadata for harvesting by other tools and services
* `LICENSE`: the (main) license of the project
* `LICENSES`: copies of all licenses that apply to files in the project
* `.reuse/dep5`: granular license and copyright information for all files and directories

*Development:*

* `pyproject.toml`: project metadata, dependencies, development tool configurations
* `poetry.lock`: needed for reproducible installation of the project
* `src`: actual code provided by the project
* `tests`: all tests for the code in the project
* `mkdocs.yml`: configuration of the project website
* `docs`: most contents used for the project website

*Automation and Quality Control:*

* `.pre-commit-config.yaml`: quality assurance tools used in the project
* `.github/workflows`: CI scripts for GitHub (QA, documentation and package deployment)
* `.github/ISSUE_TEMPLATE`: templates for the GitHub issue tracker
* `.gitlab-ci.yml`: mostly equivalent CI scripts, but for GitLab
* `.gitlab/issue_templates`: The same issues templates, but for GitLab
=== "General"
* `AUTHORS.md`: acknowledges and lists all contributors
* `CHANGELOG.md`: summarizes the changes for each version of the software for users
* `CODE_OF_CONDUCT.md`: defines the social standards that must be followed by contributors
* `CONTRIBUTING.md`: explains how others can contribute to the project
* `README.md`: provides an overview and points to other resources

=== "Metadata"
* `CITATION.cff`: metadata stating how to cite the project
* `codemeta.json`: metadata for harvesting by other tools and services
* `LICENSE`: the (main) license of the project
* `LICENSES`: copies of all licenses that apply to files in the project
* `.reuse/dep5`: granular license and copyright information for all files and directories

=== "Development"
* `pyproject.toml`: project metadata, dependencies, development tool configurations
* `poetry.lock`: needed for reproducible installation of the project
* `src`: actual code provided by the project
* `tests`: all tests for the code in the project
* `mkdocs.yml`: configuration of the project website
* `docs`: most contents used for the project website

=== "CI / QA"
* `.pre-commit-config.yaml`: quality assurance tools used in the project
* `.github/workflows`: CI scripts for GitHub (QA, documentation and package deployment)
* `.github/ISSUE_TEMPLATE`: templates for the GitHub issue tracker
* `.gitlab-ci.yml`: mostly equivalent CI scripts, but for GitLab
* `.gitlab/issue_templates`: The same issues templates, but for GitLab

!!! tip
You might find various other files popping up which are generated by different tools.
Most of these should not be committed into the repository, so they are excluded
in the `.gitignore` file. Everything listed there is safe to delete.

### Used Tools

Here is a *non-exhaustive* list of the most important tools used in the project.

Best practices for modern Python development are implemented by using:

* `poetry` for dependency management and packaging
* `pytest` for unit testing
* `hypothesis` for property-based testing
* `pre-commit` for orchestrating linters, formatters and other utilities
* `black` for source-code formatting
* `autoflake` for automatically removing unused imports
* `flake8` for general linting (using various linter plugins)
* `pydocstyle` for checking docstring conventions
* `interrogate` for computing docstring coverage
* `mypy` for editor-independent type-checking
* `mkdocs` for generating the project documentation website
* `bandit` for checking security issues in the code
* `safety` for checking security issues in the current dependencies

Metadata best practices for FAIR software are implemented using:

* `cffconvert` to check the `CITATION.cff` (citation metadata)
* `codemetapy` to generate a `codemeta.json` (general software metadata)
* `somesy` to keep all important metadata continuously synchronized
* `reuse` to check [REUSE-compliance](https://reuse.software/spec/) (granular copyright and license metadata)
* `licensecheck` to scan for possible license incompatibilities in the dependencies
=== "General"
* `poetry` for dependency management and packaging
* `poethepoet` tool for running common tasks
* `pre-commit` for orchestrating linters, formatters and other utilities
* `mkdocs` for generating the project documentation website
* `mike` for managing the `mkdocs`-generated documentation website

=== "Code Quality"
* `flake8` for general linting (using various linter plugins)
* `mypy` for editor-independent type-checking
* `pytest` for unit testing
* `pytest-cov` for computing code coverage by tests
* `hypothesis` for property-based testing
* `bandit` for checking security issues in the code
* `safety` for checking security issues in the current dependencies

=== "Formatting and Style"
* `black` for source-code formatting
* `autoflake` for automatically removing unused imports
* `pydocstyle` for checking docstring conventions

=== "FAIR metadata"
* `cffconvert` to check the `CITATION.cff` (citation metadata)
* `codemetapy` to generate a `codemeta.json` (general software metadata)
* `somesy` to keep all important metadata continuously synchronized
* `reuse` to check [REUSE-compliance](https://reuse.software/spec/) (granular copyright and license metadata)
* `licensecheck` to scan for possible license incompatibilities in the dependencies

!!! tip
Most tools installed and used by this project are listed in the
`pyproject.toml` and `.pre-commit-config.yaml` files.

## Basics

Expand All @@ -85,8 +94,9 @@ In older software, most of this information is often scattered over many little
tool-specific configuration files and a `setup.py`, `setup.cfg` and/or `requirements.txt`
file.

In this project, `pyproject.toml` is the first place that should be checked
when looking for the configuration of some development tool.
!!! tip
`pyproject.toml` is the first place your should check
when looking for the configuration of some development tool.

### Configuration

Expand All @@ -109,8 +119,10 @@ Note that `poetry` is only needed for development of the repository.
The *end-users* who just want to *install and use* this project
do **not need** to set up or know anything about poetry.

Note that if you use `poetry shell` and the project is installed with `poetry install`,
in the following you do not have to prepend `poetry run` to commands you will see below.
!!! tip
If you use `poetry shell` to activate the virtual environment of the project,
and the project is already installed with `poetry install`, in the following you do not
have to prepend `poetry run` in the commands you will see below.

### Task Runner

Expand All @@ -123,7 +135,7 @@ Often projects use a shell script or `Makefile` for this purpose. This project u
[poethepoet](https://github.com/nat-n/poethepoet), as it integrates nicely with `poetry`.
The tasks are defined in `pyproject.toml` and can be launched using:

```
```bash
poetry run poe TASK_NAME
```

Expand Down Expand Up @@ -153,7 +165,7 @@ be activated. This is usually done using `pre-commit install`, which also requir

In this project, you can run:

```
```bash
poetry run poe init-dev
```

Expand Down Expand Up @@ -182,7 +194,7 @@ After doing that, you can retry to `git commit` your changes.
To avoid having to deal with many issues at once, it is a good habit to run
`pre-commit` by hand from time to time. In this project, this can be done with:

```
```bash
poetry run poe lint --all-files
```

Expand All @@ -204,10 +216,14 @@ Such randomized tests can be a good addition to hand-crafted tests and inputs.

To run all tests, either invoke `pytest` directly, or use the provided task:

```
```bash
poetry run poe test
```

!!! tip
Add the flag `--cov` to enable the test coverage tracking and get a table with
results after the tests are completed.

## Documentation

The project uses [`mkdocs`](https://www.mkdocs.org/) with the popular and excellent
Expand All @@ -234,6 +250,11 @@ To make this both possible as well as convenient, this project uses
[`mike`](https://github.com/jimporter/mike) to generate and manage the `mkdocs`
**documentation for different versions** of the software.

!!! tip
You can easily add new pages (e.g. extended tutorials or topic-specific guides) to
your documentation website by creating markdown files in the `docs/` directory and
adding them to the `nav` section in `mkdocs.yml`.

### Online Documentation

To avoid dependence on additional services such as [readthedocs](https://readthedocs.org/),
Expand All @@ -243,19 +264,31 @@ The provided CI pipeline automatically generates the documentation for the lates
development version (i.e., current state of the `main` branch) as well as every released
version (i.e., marked by a version tag `vX.Y.Z`).


!!! tip
Should anything go wrong and you need to manually access the data of the deployed
website, you can find it in the `gh-pages` branch of the repository. Normally you
should not need to use that branch directly, though.


### Offline Documentation

You can manually generate a local and fully offline copy of the documentation, which
can be useful for e.g. previewing the results during active work on the documentation:

```
```bash
poetry install --with docs
poetry run poe docs
```

Once the documentation site is built, run `mkdocs serve` and
open `https://localhost:8000` in your browser to see the local copy of the website.

!!! tip
You probably should always check bigger website updates locally before it is publicly
deployed. The automatic pipelines can only catch technical problems, but you still
e.g. might want to do some proof-reading.

## Releases

From time to time the project is ready for a new **release** for users.
Expand Down Expand Up @@ -315,4 +348,4 @@ in the `publish` job in `ci.yml` to enable the corresponding publication target.

If the old and less secure token-based authentication method is needed or
the package should be published to a different PyPI-compatible package index, please
adapt `release.yml` [accordingly](https://github.com/pypa/gh-action-pypi-publish)).
adapt `release.yml` [accordingly](https://github.com/pypa/gh-action-pypi-publish).
2 changes: 1 addition & 1 deletion {{ cookiecutter.__project_slug }}/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ build-backend = "poetry.core.masonry.api"

# NOTE: You can run the following with "poetry poe TASK"
[tool.poe.tasks]
init-dev = { shell = "pre-commit install" }
init-dev = "pre-commit install" # run once after clone to enable various tools
lint = "pre-commit run" # pass --all-files to check everything
test = "pytest" # pass --cov to also collect coverage info
docs = "mkdocs build" # run this to generate local documentation
Expand Down

0 comments on commit ea16806

Please sign in to comment.