Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📚 Add documentation and support for its deployment to GitHub Pages #181

Merged
merged 31 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
aa8c07f
Merge branch 'main' into development
github-actions[bot] Jul 17, 2024
7fe85f7
ci(.pre-commit-config.yaml): pre-commit autoupdate
pre-commit-ci[bot] Oct 28, 2024
733035a
build(deps): bump cryptography
dependabot[bot] Sep 4, 2024
7c6b7c7
build(environment.yml): update cryptography in conda environment
Michele-Alberti Nov 2, 2024
44c21d5
ci(.pre-commit-config.yaml): update pre-commit config
Michele-Alberti Nov 3, 2024
2a5102d
docs: improve docstrings and adopt google docstring format
Michele-Alberti Nov 3, 2024
b033947
docs: add sphinx to project to build documentation static site
Michele-Alberti Nov 3, 2024
b5951a1
build(.dockerignore): add docs folder to .dockerignore
Michele-Alberti Nov 3, 2024
66d3b8d
build(makefile): improve clean-folders rule
Michele-Alberti Nov 3, 2024
46ece93
refactor(__version__): move dlunch __version__ from __init__.py to co…
Michele-Alberti Nov 4, 2024
036b1a9
docs(conf.py): add github url
Michele-Alberti Nov 4, 2024
415cec2
docs: change docs engine to mkdocs (previously was sphinx)
Michele-Alberti Nov 11, 2024
630972f
ci: add step name in ci_cd.yaml and improve 'Set Git config' step in …
Michele-Alberti Nov 11, 2024
3c83d2b
build(makefile): improve clean-folders rule
Michele-Alberti Nov 11, 2024
d96fa72
docs: add mike for mkdocs
Michele-Alberti Nov 11, 2024
5274854
build: add docs optional dipendencies to pyproject.toml
Michele-Alberti Nov 11, 2024
b2a40a8
build(pyproject.toml): fix optional-dependencies to be dynamically ev…
Michele-Alberti Nov 12, 2024
a927770
ci(deploy_docs.yaml): add workflow to deploy docs with mkdocs and mike
Michele-Alberti Nov 12, 2024
3e2c9c0
ci(deploy_docs.yaml): change latest from alias to identifier
Michele-Alberti Nov 12, 2024
35d1c49
ci(.pre-commit-config.yaml): pre-commit autoupdate
pre-commit-ci[bot] Nov 11, 2024
c9c2ff5
build(deps): bump commitizen-tools/commitizen-action
dependabot[bot] Nov 11, 2024
1896fc2
build(makefile): improve commands to add serve with mike
Michele-Alberti Nov 12, 2024
1194220
docs: improve index.md and fix mkdocs.yml to work with mike
Michele-Alberti Nov 12, 2024
c042a53
build(makefile): remove echo for docs serve rules
Michele-Alberti Nov 12, 2024
4839f77
build(add-dev-requirements): add jupyter lab to development requirements
Michele-Alberti Nov 12, 2024
84db787
docs(mkdocs.yml): change navbar
Michele-Alberti Nov 12, 2024
ca2ee97
docs(README.md): remove section numbers and add doc-start anchor
Michele-Alberti Nov 13, 2024
158a002
feat(generate_getting_started.py): remove text before doc-start ancho…
Michele-Alberti Nov 13, 2024
643bf0c
docs(index.md): add front matter
Michele-Alberti Nov 13, 2024
6209d5a
docs(mkdocs.yml): improve nav section and add navigation features
Michele-Alberti Nov 13, 2024
bbf39c6
docs(improve-docs-styling-with-css-rules): fix also indentation in ap…
Michele-Alberti Nov 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -439,4 +439,5 @@ dist
/.github
/.vscode
/docker
/.*
/.*
/docs/
6 changes: 3 additions & 3 deletions .github/workflows/bump_version.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
- name: Bump and changelog
id: cz
uses: commitizen-tools/commitizen-action@0.21.0
uses: commitizen-tools/commitizen-action@0.22.0
with:
github_token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
prerelease: "${{ steps.vars.outputs.pre_release }}"
Expand All @@ -56,8 +56,8 @@ jobs:
token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
- name: Set Git config
run: |
git config --local user.name "github-actions[bot]"
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "${{ github.actor }}"
git config --global user.email "${{ github.actor }}@users.noreply.github.com"
- name: Merge master back to dev
run: |
git checkout development
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/ci_cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ jobs:
name: release

steps:
- uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand Down
65 changes: 65 additions & 0 deletions .github/workflows/deploy_docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: 📚 Deploy docs

on:
push:
branches:
- development
tags:
- v*

# Makes sure only one workflow runs at a time.
concurrency:
group: 'pages'
cancel-in-progress: false

jobs:
build_deploy:
name: 🚚 Build and deploy 📗
runs-on: ubuntu-latest
environment:
name: github-pages
url: http://Michele-Alberti.github.io/data-lunch
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
- name: Set Git config
run: |
git config --global user.name "${{ github.actor }}"
git config --global user.email "${{ github.actor }}@users.noreply.github.com"
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11.7'
cache: 'pip'
- name: Install requirements
# Alternatively: pip install mkdocs
run: |
python -m pip install --upgrade pip
pip install -e ".[docs]"
- name: Get current version
id: get_version
# Get first line starting with version from pyproject.toml, read only major and minor and add a leading v
run: |
echo "get version from pyproject.toml"
echo ""
MAJOR_MINOR_VERSION=$(grep -m 1 '^version =' pyproject.toml | sed -E 's/version = "([0-9]+\.[0-9]+).*/\1/')
VERSION="v$MAJOR_MINOR_VERSION"
echo "current version (major-minor): $VERSION"
echo "version=$VERSION" >> $GITHUB_ENV
- name: Build and deploy documentation for development
if: github.ref_name == 'development'
run: |
mike deploy --push --update-aliases latest
- name: Build and deploy documentation for tagged version
if: startsWith(github.event.ref, 'refs/tags/v')
run: |
ALIAS=stable
echo "version: $VERSION"
echo "alias: $ALIAS"
mike deploy --push --update-aliases "$VERSION" "$ALIAS"
- name: Update default (stable)
if: ${{ env.alias }} == 'stable'
run: mike set-default --push stable
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -439,4 +439,5 @@ dist

/shared_data/
/ssl/
*.csv
*.csv
/docs/api/
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
repos:
- repo: https://github.com/psf/black
rev: 24.4.2
rev: 24.10.0
hooks:
- id: black
language_version: python3
- repo: https://github.com/pycqa/flake8
rev: 7.1.0
rev: 7.1.1
hooks:
- id: flake8
- repo: https://github.com/commitizen-tools/commitizen
rev: v3.27.0
rev: v3.30.1
hooks:
- id: commitizen
stages: [commit-msg]
- repo: https://github.com/kynan/nbstripout
rev: 0.7.1
rev: 0.8.0
hooks:
- id: nbstripout
default_install_hook_types: [pre-commit, commit-msg]
Expand Down
99 changes: 52 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,41 @@

The ultimate web app for a well organized lunch.

## 1. Table of contents

- [1. Table of contents](#1-table-of-contents)
- [2. Development environment setup](#2-development-environment-setup)
- [2.1. Miniconda](#21-miniconda)
- [2.2. Setup the development environment](#22-setup-the-development-environment)
- [2.3. Environment variables](#23-environment-variables)
- [2.3.1. General](#231-general)
- [2.3.2. Docker and Google Cloud Platform](#232-docker-and-google-cloud-platform)
- [2.3.3. TLS/SSL Certificate](#233-tlsssl-certificate)
- [2.3.4. Encryption and Authorization](#234-encryption-and-authorization)
- [2.4. Manually install the development environment](#24-manually-install-the-development-environment)
- [2.5. Manually install data-lunch CLI](#25-manually-install-data-lunch-cli)
- [2.6. Running the docker-compose system](#26-running-the-docker-compose-system)
- [2.7. Running a single container](#27-running-a-single-container)
- [2.8. Running locally](#28-running-locally)
- [3. Additional installations before contributing](#3-additional-installations-before-contributing)
- [3.1. Pre-commit hooks](#31-pre-commit-hooks)
- [3.2. Commitizen](#32-commitizen)
- [4. Release strategy from `development` to `main` branch](#4-release-strategy-from-development-to-main-branch)
- [5. Google Cloud Platform utilities](#5-google-cloud-platform-utilities)

## 2. Development environment setup
## Table of contents

- [Table of contents](#table-of-contents)
- [Development environment setup](#development-environment-setup)
- [Miniconda](#miniconda)
- [Setup the development environment](#setup-the-development-environment)
- [Environment variables](#environment-variables)
- [General](#general)
- [Docker and Google Cloud Platform](#docker-and-google-cloud-platform)
- [TLS/SSL Certificate](#tlsssl-certificate)
- [Encryption and Authorization](#encryption-and-authorization)
- [Manually install the development environment](#manually-install-the-development-environment)
- [Manually install data-lunch CLI](#manually-install-data-lunch-cli)
- [Running the docker-compose system](#running-the-docker-compose-system)
- [Running a single container](#running-a-single-container)
- [Running locally](#running-locally)
- [Additional installations before contributing](#additional-installations-before-contributing)
- [Pre-commit hooks](#pre-commit-hooks)
- [Commitizen](#commitizen)
- [Release strategy from `development` to `main` branch](#release-strategy-from-development-to-main-branch)
- [Google Cloud Platform utilities](#google-cloud-platform-utilities)

<!-- DO NOT REMOVE THIS ANCHOR -->
<!-- Used by MkDocs generate_getting_started.py -->
<a id="doc-start"></a>

## Development environment setup

The following steps will guide you through the installation procedure.

### 2.1. Miniconda
### Miniconda

[Conda](https://docs.conda.io/en/latest/) is required for creating the development environment (it is suggested to install [Miniconda](https://docs.conda.io/en/latest/miniconda.html)).

### 2.2. Setup the development environment
### Setup the development environment

Use the setup script (`setup_dev_env.sh`) to install all the required development tools.

Expand All @@ -49,19 +53,19 @@ source setup_dev_env.sh
> - _pre-commit_ hooks
> - `data-lunch` command line

### 2.3. Environment variables
### Environment variables

The following environment variables are required for running the _web app_, the _makefile_ or _utility scripts_.

#### 2.3.1. General
#### General
| Variable | Type | Required | Description |
|----------|:----:|:--------:|-------------|
`PANEL_APP` | _str_ | ✔️ | app name, _data-lunch-app_ by default (used by `makefile`)
`PANEL_ENV` | _str_ | ✔️ | environment, e.g. _development_, _quality_, _production_, affects app configuration (_Hydra_) and build processes (_makefile_)
`PANEL_ARGS` | _str_ | ❌ | additional arguments passed to _Hydra_ (e.g. `panel/gui=major_release`) in _makefile_ and `docker-compose` commands
`PORT` | _int_ | ✔️ | port used by the web app (or the container), default to _5000_; affects app configuration and build process (it is used by _makefile_, _Hydra_ and _Docker_)

#### 2.3.2. Docker and Google Cloud Platform
#### Docker and Google Cloud Platform
> [!NOTE]
> The following variables are mainly used during the building process or by external scripts.

Expand All @@ -76,7 +80,7 @@ The following environment variables are required for running the _web app_, the
`MAIL_RECIPIENTS` | _str_ | ❌ | email recipients as string, separated by `,` (used for sending emails containing the instance IP when hosted on _Google Cloud Platform_)
`DUCKDNS_URL` | _str_ | ❌ | _URL_ used in `compose_init.sh` to update dynamic address (see _Duck DNS's_ instructions for details, used when hosted on _Google Cloud Platform_)

#### 2.3.3. TLS/SSL Certificate
#### TLS/SSL Certificate
> [!TIP]
> Use the command `make ssl-gen-certificate` to generate local SSL certificates.

Expand All @@ -85,7 +89,7 @@ The following environment variables are required for running the _web app_, the
`CERT_EMAIL` | _str_ | ❌ | email for registering _SSL certificates_, shared with the authority _Let's Encrypt_ (via `certbot`); used by `docker-compose` commands
`DOMAIN` | _str_ | ❌ | domain name, e.g. _mywebapp.com_, used by `docker-compose` commands (`certbot`), in email generation (`scripts` folder) and to auto-generate SSL certificates

#### 2.3.4. Encryption and Authorization
#### Encryption and Authorization
> [!NOTE]
> All variables used by _Postgresql_ are not required if `db=sqlite` (default value).
> All variables used by _OAuth_ are not required if `server=no_auth` (default value).
Expand All @@ -108,13 +112,13 @@ The following environment variables are required for running the _web app_, the
`DATA_LUNCH_DB_DATABASE` | _str_ | ❌ | _Postgresql_ database, do not set to use default value
`DATA_LUNCH_DB_SCHEMA` | _str_ | ❌ | _Postgresql_ schema, do not set to use default value

### 2.4. Manually install the development environment
### Manually install the development environment
> [!WARNING]
> This step is not required if the [setup script](#22-setup-the-development-environment) (`setup_dev_env.sh`) is used.
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

Use the terminal for navigating to the repository base directory.\
Use the following command in your terminal to create an environment named `data-lunch` manually.
Otherwise use the [setup script](#22-setup-the-development-environment) to activate the guided installing procedure.
Otherwise use the [setup script](#setup-the-development-environment) to activate the guided installing procedure.

```
conda env create -f environment.yml
Expand All @@ -126,10 +130,10 @@ Activate the new _Conda_ environment with the following command.
conda activate data-lunch
```

### 2.5. Manually install data-lunch CLI
### Manually install data-lunch CLI

> [!WARNING]
> This step is not required if the [setup script](#22-setup-the-development-environment) (`setup_dev_env.sh`) is used.
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

The CLI is distributed with setuptools instead of using Unix shebangs.
It is a very simple utility to initialize and delete the app database. There are different use cases:
Expand Down Expand Up @@ -160,7 +164,7 @@ data-lunch --version
data-lunch --help
```

### 2.6. Running the docker-compose system
### Running the docker-compose system

Since this app will be deployed with an hosting service a _Dockerfile_ to build a container image is available.
The docker compose file (see `docker-compose.yaml`) deploys the web app container along with a _load balancer_ (the _nginx_ container)
Expand Down Expand Up @@ -189,7 +193,7 @@ You can then access your web app at `http://localhost:PORT` (where `PORT` will m
> [!IMPORTANT]
> An additional container named `db` is started if `db=postgresql` is set

### 2.7. Running a single container
### Running a single container

It is possible to launch a single server by calling the following command.

Expand All @@ -201,7 +205,7 @@ make docker-run

You can then access your web app at `http://localhost:5000` (if the deafult `PORT` is selected).

### 2.8. Running locally
### Running locally

Launch a local server with default options by calling the following command.

Expand All @@ -217,10 +221,10 @@ python -m dlunch server=basic_auth

See [Hydra's documentation](https://hydra.cc/docs/intro/) for additional details.

## 3. Additional installations before contributing
## Additional installations before contributing

> [!WARNING]
> This step is not required if the [setup script](#22-setup-the-development-environment) (`setup_dev_env.sh`) is used.
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

Before contributing please create the `pre-commit` and `commitizen` environments.

Expand All @@ -230,9 +234,10 @@ conda env create -f pre-commit.yml
conda env create -f commitizen.yml
```

### 3.1. Pre-commit hooks
### Pre-commit hooks

> This step is not required if the [setup script](#23-setup-the-development-environment-by-using-the-setup-script) is used.
> [!WARNING]
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

Then install the precommit hooks.

Expand All @@ -248,10 +253,10 @@ Optionally run hooks on all files.
pre-commit run --all-files
```

### 3.2. Commitizen
### Commitizen

> [!WARNING]
> This step is not required if the [setup script](#22-setup-the-development-environment) (`setup_dev_env.sh`) is used.
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

The _Commitizen_ hook checks that rules for _conventional commits_ are respected in commits messages.
Use the following command to enjoy _Commitizen's_ interactive prompt.
Expand All @@ -263,7 +268,7 @@ cz commit

`cz c` is a shorther alias for `cz commit`.

## 4. Release strategy from `development` to `main` branch
## Release strategy from `development` to `main` branch

> [!CAUTION]
> This step is required only if the CI-CD pipeline on _GitHub_ does not work.
Expand Down Expand Up @@ -303,9 +308,9 @@ git merge main --no-ff

Use _"update files from last release"_ or the default text as commit message.

## 5. Google Cloud Platform utilities
## Google Cloud Platform utilities

> [!WARNING]
> This step is not required if the [setup script](#22-setup-the-development-environment) (`setup_dev_env.sh`) is used.
> This step is not required if the [setup script](#setup-the-development-environment) (`setup_dev_env.sh`) is used.

The makefile has two rules for conviniently setting up and removing authentication credentials for _Google Cloud Platform_ command line interface: `gcp-config` and `gcp-revoke`.
Loading
Loading