Merge pull request #6 from ADACS-Australia/refactor/base

Update documentation

docs/conf.py

@@ -40,7 +40,8 @@ def format_authors(authors):
 copyright_year = 2023
 copyright = f"{copyright_year}, {author}"
 
-# These versions are just placeholders.  The point of truth for versions is the last tag on the main branch with a "vX.Y.Z*" format.  This can
+# These versions are just placeholders.  The point of truth for versions is
+# the last tag on the main branch with a "vX.Y.Z*" format.  This can
 # be accessed through installed project metadata.
 
 # The short X.Y.Z version
@@ -77,7 +78,8 @@ def format_authors(authors):
     "dollarmath",
 ]
 
-# Set the prompt text that you want the docs copy button to ignore when copying code cells
+# Set the prompt text that you want the docs copy button to ignore when
+# copying code cells
 copybutton_prompt_text = "$ "
 
 # Add any paths that contain templates here, relative to this directory.

docs/content/configuring_services.md

@@ -7,7 +7,13 @@ Develpers and project owners/maintainers will require accounts with one or all o
 
     To work with this codebase, you will require a *GitHub* account ([go here to get one](https://github.com)).
 
-    Branch permissions for the main project repository should be configured only permit merges from pull requests.  To do so, navigate to `Settings->Branches->Add branch ruleset` and:
+    ::: {note}
+    If you have just rendered a new project and have followed the instructions that were given, you should already have done the
+    following.  For completeness though - or in case you are looking to configure a fork of a project - we note that the following
+    needs to be done:
+    :::
+
+    Branch permissions for the main project repository should be configured to only permit merges from pull requests.  To do so, navigate to `Settings->Branches->Add branch ruleset` and:
 
     - give the Ruleset whatever name you'd like (e.g. `Protect Main`)
     - set `Enforecement status` to `Active`
@@ -17,7 +23,12 @@ Develpers and project owners/maintainers will require accounts with one or all o
 
     This will ensure that all CI/CD tests pass before a merge to the main branch can be made.
 
-    Several secrets need to be configured by navigating to `Settings->Secrets->Actions` and adding the following:
+    ::: {note}
+    For those who have just rendered a new project, followed the instructions and are continuing to configure it, you need to do
+    the following:
+    :::
+
+    Several (optional) secrets need to be configured by navigating to `Settings->Secrets->Actions` and adding the following:
 
     - To host the project documentation on *Read the Docs* (see below), the following secrets need to be set (see below for where to find these values):
 
@@ -35,12 +46,16 @@ Develpers and project owners/maintainers will require accounts with one or all o
 
 2. [__Read the Docs__](https://readthedocs.org)
 
-    **Read the Docs** (*RTD*) is used to build and host the project documentation.  An account is needed if you are an owner/maintainer of the project and will be publishing and managing the project's documentation online, but not needed if you are simply a contributing developer.  *RTD* can be configured in either of the following ways:
+    ::: {note}
+    **Read the Docs** (*RTD*) is used to build and host the project documentation.  To take advantage of this functionality, this service needs to be configured by the owner/maintainer of the project.  Contributing developers or those who don't wish to have documentation published on *RTD* can ignore the following.
+    :::
+
+    To (optionally) publish this project's documentation you will need a *RTD* account ([go here to get one](https://readthedocs.org)).  It can be configured in either of the following ways:
 
     1. **By connecting *RTD* to your *GitHub* account**
-        - Ensure that your *GitHub* account has been connected.  This is done automatically if you log into *RTD* with your *GitHub* credentials.  If you logged in with your email, navigate to `<login_id>->Settings->Connected Services` by clicking on "Connect Your Accounts" and click "Connect to GitHub".  You know your account is linked if it is listed below under "Active Services".
+        - Ensure that your *GitHub* account has been connected.  This is done automatically if you log into *RTD* with your *GitHub* credentials.  If you logged in with your email, navigate to `<login_id>->Settings->Connected Services` by clicking on `Connect Your Accounts` and click `Connect to GitHub`.  You know your account is linked if it is listed below under `Active Services`.
 
-        - Return to your *RTD* landing page by clicking on your account name at the top.  Click "Import a Project".  Your *GitHub* repository should be listed here (you may need to refresh the list if it has been created recently).  Import it.
+        - Return to your *RTD* landing page by clicking on your account name at the top.  Click `Import a Project`.  Your *GitHub* repository should be listed here (you may need to refresh the list if it has been created recently).  Import it.
 
         - To obtain **RTD_WEBHOOK_TOKEN**, navigate to `<Account>->Settings->API Tokens` on *Read the Docs*.  If a token has been created already, you can use it.  Otherwise (or if you want a token specifically for this project), create a new one.
 
@@ -60,9 +75,19 @@ Develpers and project owners/maintainers will require accounts with one or all o
 (config-pypi)=
 3. The [__Python Package Index (*PyPI*)__](https://pypi.org)
 
-    This service is used to publish project releases.  An account is needed if you are the owner of the project, but not generally needed if you are simply a contributing developer.  An API token will need to be created and added to your *GitHub* project as **PYPI_TOKEN** (as detailed above).  This can be generated from the *PyPI* UI by navigating to `Account Settings->Add API Token`.
+    ::: {note}
+    **The Python Package Index** (*PyPI*) and **The Test Python Package Index (*TestPyPI*) are used to publish code releases.  To take advantage of this functionality, this service needs to be configured by the owner/maintainer of the project.  Contributing developers or those who don't wish to have documentation published on *RTD* can ignore the following.
+    :::
+
+    To (optionally) publish releases of this project's code you will need a *PyPI* account ([go here to get one](https://pypi.org)).  To (optionally) generate test releases, you will need a *TestPyPI* account.  They operate the same way and can both be configured as follows:
 
-    To test releases, a parallel account on *test.PyPI* is needed and a similar token to **PYPI_TOKEN** - named **TEST_PYPI_TOKEN** needs to be set, in the same way as above.  To create a test release, flag it as a "pre-release" through the *GitHub* interface when you generate a release, and it will be published on *test.PyPI.org* rather than *PyPI.org*.  If this token is not defined, publication will not happen on either.
+        - An API token will need to be created and added to your *GitHub* project as **PYPI_TOKEN** (as detailed above).  This can be generated from the *PyPI* UI by navigating to `Account Settings->Add API Token`.  In the first instance - before the project exists on your account - generate a token with `Entire Account` scope.  Once the project has been published for a first time to *PyPI*, the initial token can be deleted and a new one generated with an project scope selected to be that of the project.  Make sure to update the *GitHub* secret value once you have done so.
+
+        - Repeat this process for *TestPyPI* if you want to be able to generate test releases, adding it as a secret to your *GitHub* repo with the name **TEST_PYPI_TOKEN**
+
+    ::: {note}
+    To create a test release, flag it as a `pre-release` through the *GitHub* interface when you generate a release, and it will be published on *test.PyPI.org* rather than *PyPI.org*.  If this token is not defined, publication will not happen on either, if this option is flagged.
+    :::
 
     ::: {note}
     Although `poetry` can be used to directly publish this project to *PyPI*, users should not do this.  The proper way to publish the project is through the *GitHub* interface, which leverages the *GitHub Workflows* of this project to ensure the enforcement of project standards before a new version can be created.

docs/content/notes_for_developers.md → docs/content/development_guidelines.md

@@ -1,6 +1,8 @@
-# Notes for Developers
+(developing-new)=
+# Guidelines for Developing a New Project
 
-This section provides details on how to configure and/or develop this codebase.
+This section provides guidelines that should be followed when developing a new project with this template.  It goes through what
+gets set-up when you render a new project and explains why things have been set-up they way they have.
 
 ## Continuos Integration/Continuous Deployment (CI/CD) Workflow
 
@@ -243,12 +245,11 @@ The majority of documentation changes can be managed in one of the following 4 w
 
 4. **Add a new Markdown file**:
 
-	Otherwise, create a new `.md` file in the `docs/content` directory and add it to the list of Markdown files referenced in `docs/index.rst`.  Note that these files will be added to the documentation in the order specified, so place it in that list where you want it to appear in the final documentation.  This new `.md` file should start with a top-level title (marked-up by starting a line with a single `#`; see the top of this file for an example).
+    Otherwise, create a new `.md` file in the `docs/content` directory and add it to the list of Markdown files referenced in `docs/index.rst`.  Note that these files will be added to the documentation in the order specified, so place it in that list where you want it to appear in the final documentation.  This new `.md` file should start with a top-level title (marked-up by starting a line with a single `#`; see the top of this file for an example).
 
-5. **Extend the MyST-Parser support***
+5. **Extend the MyST-Parser support**:
 
-    New MyST-Parser extensions can be enabled in `docs/conf.py` by extending the `myst_enable_extensions` list.  See the
-    MyST-Parser documentation for a list of available extensions and instructions on how to use them.
+    New MyST-Parser extensions can be enabled in `docs/conf.py` by extending the `myst_enable_extensions` list.  See the MyST-Parser documentation for a list of available extensions and instructions on how to use them.
 
 #### Adding images, etc.
 

docs/content/template_maintenance.md

@@ -0,0 +1,29 @@
+# Maintaining This Template
+
+Developers maintaining this template codebase should be aware of the development
+guidelines presented [here](#developing-new).  This section discusses issues specific to
+the maintenance of this template.
+
+## Running tests
+
+When testing, temporary projects get created and need to be installed into an environment before tests can be run, documentation builds checked, etc.  Be warned that this can both pollute your development environment and lead to unreliable tests due to the bleeding of state from past test runs to new runs.
+
+To address this we will soon add the ability to run tests in a Docker container, where we can ensure fresh and reproducable test runs.  This is not implemented yet but will be soon.
+
+## Some things to note about Cookiecutter templates
+
+The following are some things that you might not know about Cookiecutter templates that you may need to be aware of if you are
+developing on this template:
+
+### Hooks
+
+There are two files in the `hooks` directory: one which gets run *before* a tempalte gets rendered and one which gets run *after* a template gets rendered.  **Both of these files are actually Jinja2 templates.**  As a result, these files can legitimately possess invalid code.  **Be particularly careful if your code editor is configured to autoformat code, since this can cause unintended bugs in these files.**
+
+## Changes to documentation
+
+Make sure that any changes that get made to the template are reflected in the
+documentation.  Things to pay attention to are:
+
+1. The project and template READMEs
+2. The INSTRUCTIONS.template file
+3. The documentation in the template and this documentation here

docs/index.rst

@@ -9,5 +9,6 @@
    :maxdepth: 3
 
    Home <self>
-   Configuring a New Project <content/configuring_services>
-   Development Guidelines <content/notes_for_developers>
+   Configuring a new project <content/configuring_services>
+   Development guidelines <content/development_guidelines>
+   Maintianing this template <content/template_maintenance>

hooks/post_gen_project.py

@@ -8,8 +8,10 @@
 def git() -> None:
     """Create and configure a git repo for the rendered project"""
     result = subprocess.run(["git", "init", "-q",  "-b", "main"])
-    result = subprocess.run(["git", "config",  "--local", "push.followTags", "true" ])
-    result = subprocess.run(["git", "config",  "--local", "--add", "remote.origin.tagopt", "--tags" ])
+    result = subprocess.run(
+        ["git", "config",  "--local", "push.followTags", "true"])
+    result = subprocess.run(
+        ["git", "config",  "--local", "--add", "remote.origin.tagopt", "--tags"])
     result = subprocess.run(
         [
             "git",
@@ -33,11 +35,13 @@ def git() -> None:
 
     # Create initial commit
     result = subprocess.run(["git", "add", "*"])
-    result = subprocess.run(["git", "commit", "-q", "-m", "Initial commit of template code"])
+    result = subprocess.run(
+        ["git", "commit", "-q", "-m", "Initial commit of template code"])
 
     # Make sure this is an annotated tag or it won't be pushed to remote and the version
     # management won't be initialised correctly
-    result = subprocess.run(["git", "tag", "-a", "v0.0.0", "-m", "Initialised with template"])
+    result = subprocess.run(
+        ["git", "tag", "-a", "v0.0.0", "-m", "Initialised with template"])
 
 
 def venv(venv_type: str) -> None:
@@ -85,7 +89,8 @@ def venv(venv_type: str) -> None:
     elif venv_type == "none":
         pass
     else:
-        raise ValueError(f"Invalid venv implementation selection given: {venv_type}")
+        raise ValueError(
+            f"Invalid venv implementation selection given: {venv_typ}")
 
 
 def install(venv_type: str) -> None:
@@ -101,11 +106,11 @@ def install(venv_type: str) -> None:
     elif venv_type == "none":
         pass
     else:
-        raise ValueError(f"Invalid venv implementation selection given: {venv_type}")
+        raise ValueError(
+            f"Invalid venv implementation selection given: {venv_type}")
 
 
 def print_instructions() -> None:
-
     """Print instructions about how to configure the rendered project for use"""
     # Read the instructions file, ignoring lines that start with '***'
     with open("INSTRUCTIONS.template", "r") as file:
@@ -122,7 +127,7 @@ def print_instructions() -> None:
     # Render the file as Markdown with Rich
     print()
     console = Console()
-    if not {{ cookiecutter.__test }}:
+    if not {{cookiecutter.__test}}:
         console.print(Markdown(lines))
 
 

pyproject.toml

@@ -46,6 +46,10 @@ addopts = "--ignore={{cookiecutter.repo_name}}"
 [tool.mypy]
 ignore_missing_imports = true
 mypy_path = "$MYPY_CONFIG_FILE_DIR/tests"
+exclude = [
+  '\{\{cookiecutter.repo_name\}\}', # Jinja2 templates which may have invalid Python code
+  'hooks' # These files are treated as Jinja2 templates and may have invalid Python code
+]
 
 [tool.black]
 extend-exclude = '''
@@ -73,7 +77,7 @@ ignore = [
 ]
 
 # Ignore `E402` (import violations) in all `__init__.py` files, and in `path/to/file.py`.
-[tool.ruff.per-file-ignores]
+[tool.ruff.lint.per-file-ignores]
 "__init__.py" = ["E402"]
 
 [tool.ruff.lint.pydocstyle]

tests/test_bake_project.py

@@ -11,7 +11,6 @@ def test_year_in_license_file(cookies):
 
 
 def test_bake_with_defaults(cookies):
-
     # List of filenames that need to be present in a properly rendered template
     check_toplevel_pathnames = [
         ".git",

{{cookiecutter.repo_name}}/INSTRUCTIONS.template

@@ -4,58 +4,59 @@
 
 *Now that your template has been rendered, we recommend configuring it as follows:*
 
-1. **Create a Python environment**
+1. **Create and activate a Python 3.11 environment**
 
-Use your favourite method to create and activate a Python environment for your development.
+Use your favourite method to create and activate a Python 3.11 environment for your development.
 
 2. **Install the project and all dependencies**
 
 With your environment activated, run the following from within your new repo: `poetry install`
 
-***NOTE: This needs to be done before any tests can be run, documentation built, etc.***
+*NOTE: This needs to be done before any tests can be run, documentation built, etc.*
 
-3. **Create a GitHub repo**
+3. **Create a GitHub repo and adjust workflow permissions**
 
 Goto *`https://github.com`* and create a new repo with the following:
 
 - account:   **{{cookiecutter.github_login}}**
 - repo name: **{{cookiecutter.repo_name}}**
 
-4. **Enable GitHub Actions write permissions**
+Navigate within the repo to `Settings->Actions->General` and select `Read and write permissions`.
 
-On Github, navigate within the repo to `Settings->Actions->General` and select `Read and write permissions`.
+*NOTE: This needs to be done before the next stage so that the GitHub version bump action that is run on first upload does not fail.*
 
-***NOTE: This needs to be done before the next stage so that the GitHub version bump action that is run on first upload does not fail.***
-
-5. **Push the local repo to GitHub**
+4. **Push the local repo to GitHub**
 
 Your new repo has been configured with your GitHub repo details.  Run the following from within your new local repo to push it to GitHub: `git push -u origin main`
 
-6. **Configure branch permissions**
+*NOTE: If you notice any failed workflow runs at this point, it may be because you forgot to make the Action permission change mentioned above.*
+
+5. **Configure branch permissions**
 
 Navigate to `Settings->Branches->Add branch ruleset` and ...
 
 - give the Ruleset whatever name you'd like (e.g. `Protect Main`)
 - set `Enforecement status` to `Active`
 - add a `Target Branch` targeting criteria by pattern and type `main`
 - select `Require a pull request before merging`
-- select `Require status checks to pass` and add `Run all build and unit tests` from GitHub Actions as a required check
+- select `Require status checks to pass` and add the check `Run all build and unit tests` as a required check
+- push the `Create` button at the bottom
 
-***From this point on, no development should be performed on the main branch.  All updates to the main branch are managed via Pull
-Requests in the GitHub UI.***
+*From this point on, no development should be performed on the main branch.  All updates to the main branch are managed via Pull Requests in the GitHub UI.*
 
-7. **[Optional] Activate pre-commit git hooks**
+6. **[Optional] Activate pre-commit git hooks**
 
 From within the repo, run the following: `pre-commit install`
 
-8 **[Optional] Configure Services (Read The Docs, PyPI, etc.)**
+7 **[Optional] Configure Services (Read The Docs, PyPI, etc.)**
 
 This can be done at any time when you want to enable these features; they are simply deactivated until you do; see *`https://adacs-python-template.readthedocs.io/en/latest/content/configuring_services.html`* for instructions.
 
-9. **Start developing!**
+8. **Start developing!**
 
-No development should be done on *main*, so create and checkout a new git branch with `git checkout -b new_branch_name` and start
+**No development should be done on *main***, so create and checkout a new git branch with `git checkout -b new_branch_name` and start
 developing.
 
+**N.B.: If you are new to the use of this template, make sure you have a good read of the `Notes for Developer
 If you have any questions, consult the template documentation at *`https://adacs-python-template.readthedocs.io/en/latest/index.html`*.