new version 0.2.0 (#50)

* Bump version to next major version #9

* Revert flake8 to ^3.0 for docstrings #9

* add a notebook that shows how to run init_config

* Make dev dependencies optional, update lock and README #9

* Update workflow install --with dev, add matrix poetry version #9

* Fix python and poetry version definition #9

* Fix python and poetry version definition #9

* fix lock file

* update dev dependencies and add python-dotenv to dev

* improve readme

* update readme

* Feat/8 handle multiple databases and users (#20)

* change config module to handle multiple databases

* finalize work on config module to handle multiple databases; significantly reduced lines of code by getting rid of the settings.ini

* add a new db module that serves as a layer between the user and the config. Can set the current active database and get the settings from the config

* simplify config module

* refactor code to implement new config; correct tests

* fix all remaining tests

* fix all text issues

* update notebooks according to latest changes in config

* drop support for Python 3.9 due to pipe operator for types and set supported versions to 3.10 and 3.11

* fix problem with config dir creation during setup

* fix isort

* Improve clear_cache output for full wipe, remove unused import

* Address all non global-related pylint issues #20

* because of complexity get rid of the current support of custom config dir and always use the default config dir under user home.

* fix all tests; get rid of settings.ini and functionality for user to define own config path; pystatis supports only default config path but custom data cache path

* fix all tests; get rid of settings.ini and functionality for user to define own config path; pystatis supports only default config path but custom data cache path

* refactor config module to work with a ConfigParser global config object instead of overwriting the config variable within the functions using global (bad style according to pylint)

* address pylint issues

* fix mypy issues

* fix pylint issues

---------

Co-authored-by: MarcoHuebner <marco_huebner1@gmx.de>

* update README to the latest changes of multi database support

* Added lists of all available statistics and tables

* Feat/10 update and auto deploy sphinx (#27)

* Updated dev-dependencies, added first version of Sphinx documentation, including built html documentation.

* Added Logo, updated theme, updated GitHub workflow, fixed docstrings in cache and cube. Hosting on ReadTheDocs has to be done by Owner/ CorrelAid (but can be requested and triggered that way).

* Updated urllib3 version, but everything <2.0.0 (deprecating `strict`) should be fine...

* Updated poetry as recommended in cachecontrol issue report.

* Fixed black formatting, fixed make docs (is now ran by poetry).

* Fixed linting issue, updated packages, updated make docs.

* Updated ReadMe, added developer sphinx documentation, added custom pre-commit hook and changed to hard-coded version in docs, added built documentation to artifacts, #3

* Add deployment workflow, needs Repo updates

* Update depencies for Sphinx documentation #10

* Remove redundant docu information #10

Render parts of the README.md in the respective .rst files

* Remove unused mdinclude, fix run-test py version, update pre-commit #10

* Fix dependency group for SPhinx workflow #10

* Fix docstring parameter rendering in Sphinx #10

* Fix image rendering by mimicking folder structure #10

* Add comment on warnings related to ext.napoleon #10

* Rename deploy-docs #10

* Fix black format issue in conf.py #10

* Update deploy key, add deploy trigger comment #10

* Update documentation deploy workflow #10

* Switch to matrix.os definition #10

* Fix pull_request target in deploy workflow #10

* Update poetry.lock #10

* Import package version to Sphinx docu #10

* Manually fix black formatting issue #10

* With auto-deploy working, decrease retention days #10

* Update readme and Sphinx header references #10

* Fix deploy to update files on the remote #10

* fix cube functionality: it seems like structure of QEI header part was changed as well as DQA no longer has information about axis so we assume that the order is preserved (#43)

* add jupytext and new nb for presentation

* Feat/35 implement regex matching (#44)

* Implemented regex matching, initial commit

* Added credentials check for cubes and removed all references to set_db()

* Implemented regex matching, initial commit

* Added credentials check for cubes and removed all references to set_db()

* fix tests

* refactoring Find and Result class to work with new database detection logic; because find does not use names like Table and Cube, use has to specify the database

* fix tests
---------

Co-authored-by: Michael Aydinbas <michael.aydinbas@gmail.com>
Co-authored-by: Michael Aydinbas <michael.aydinbas@new-work.se>

* add presentation nb

* remove presentation nb for now

* Feat/19 improve readability of the table format (#42)

* Reformatting the raw data tables for readability

* Adding comments

* Applied suggested changes and run code formatting

* add tests for Table

---------

Co-authored-by: Michael Aydinbas <michael.aydinbas@new-work.se>

* prepare Table so it can parse data from three different sources

* Added description and examples of Find

* implement parse logic for prettify zensus tables

* fix pylint issues

* edits on Find section

* fixing overwritten changes

* update presentation nb

* add genesis parse code for regio, too, for the moment.

* Feat/34 visualization examples (#48)

* Add 02_Geo_visualization_example.ipynb

* changed '-' to 0 instead of nan --> reproduce Simons result

* new case study in visualization notebook, integration to presentation notebook

* catch NA-values in read_csv and added Auspraegung_Code to table.py to have the unique region identifiers

---------

Co-authored-by: jkrause <jkrause123@users.noreply.github.com>

* final presentation nb and shape data; omit file check in pre-commit

* fixed typo and beautified plots in presentation.ipynb /.py

* add a first workaround for the new Zensus zip content type

* fix all tests; separate Find and Results classes into own modules

* update dependencies

* update README

* set version to 0.2

* remove Cubes from package for now; we no longer support cubes until they are requested

* fix all tests; fix all relevant nb;

* fix pylint issues

* fix mypy issues

* add documentation key

* update changelog

---------

Co-authored-by: MarcoHuebner <marco_huebner1@gmx.de>
Co-authored-by: Pia <45008571+PiaSchroeder@users.noreply.github.com>
Co-authored-by: MarcoHuebner <57489799+MarcoHuebner@users.noreply.github.com>
Co-authored-by: zosiaboro <50183305+zosiaboro@users.noreply.github.com>
Co-authored-by: Zosia Borowska <zofia.anna.borowska@gmail.com>
Co-authored-by: jkrause123 <89632018+jkrause123@users.noreply.github.com>
Co-authored-by: jkrause <jkrause123@users.noreply.github.com>

.github/workflows/deploy-docs.yaml

@@ -0,0 +1,61 @@
+name: Deploy docs
+
+on:
+  push:
+    branches:
+      - main
+      - dev
+  pull_request:
+    branches:
+      - main
+      - dev
+  workflow_dispatch:
+
+env:
+  POETRY_VERSION: 1.6.1
+
+jobs:
+  build-and-deploy:
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 10
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10"]
+        os: [ubuntu-latest]
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Run poetry image
+        uses: abatilo/actions-poetry@v2.0.0
+        with:
+            poetry-version: $POETRY_VERSION
+      - name: Install dependencies
+        run: |
+          poetry install --with dev
+      - name: Build docs
+        run: |
+          cd docs
+          poetry run make clean
+          poetry run make html
+      # create distinguished subdirectories for each branch
+      # NOTE: force_orphan will delete all files in the branch (thus main & dev could not be concurrently deployed)
+      - name: Deploy docs main
+        uses: peaceiris/actions-gh-pages@v3
+        if: github.ref == 'refs/heads/main' || (github.event_name == 'pull_request' && github.base_ref == 'main')
+        with:
+          deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
+          publish_dir: ./docs/build/html
+          publish_branch: gh-pages
+      - name: Deploy docs dev
+        uses: peaceiris/actions-gh-pages@v3
+        if: github.ref == 'refs/heads/dev' || (github.event_name == 'pull_request' && github.base_ref == 'dev')
+        with:
+          deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
+          publish_dir: ./docs/build/html
+          destination_dir: dev
+          publish_branch: gh-pages

.github/workflows/run-tests.yaml

@@ -6,48 +6,53 @@ name: Run tests
 on:
   push:
     branches:
-    - main
-    - dev
+      - main
+      - dev
   pull_request:
     branches:
-    - main
-    - dev
+      - main
+      - dev
   workflow_dispatch:
 
+env:
+  POETRY_VERSION: 1.6.1
+
 jobs:
   test:
     runs-on: ${{ matrix.os }}
     timeout-minutes: 10
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.9", "3.10"]
-        os: [ubuntu-22.04, macOS-latest, windows-latest]
+        python-version: ["3.10", "3.11"]
+        os: [ubuntu-latest, macOS-latest, windows-latest]
 
     steps:
-    - uses: actions/checkout@v3
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v3
-      with:
-        python-version: ${{ matrix.python-version }}
-    - name: Run poetry image
-      uses: abatilo/actions-poetry@v2.0.0
-      with:
-          poetry-version: 1.6.1
-    - name: Install dependencies
-      run: |
-        poetry install
-    - name: Run tests
-      run: |
-        poetry run pytest --cov=pystatis tests
+      - uses: actions/checkout@v3
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Run poetry image
+        uses: abatilo/actions-poetry@v2.0.0
+        with:
+          poetry-version: $POETRY_VERSION
+      - name: Install dependencies
+        run: |
+          poetry install --with dev
+      - name: Run tests
+        run: |
+          poetry run pytest --cov=pystatis tests
   code-quality:
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 10
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.10.13]
-        poetry-version: [1.6.1]
+        # only support specific python version, as guidelines differ beween (minor) versions
+        python-version: ["3.11.6"]
         os: [ubuntu-latest]
-    runs-on: ${{ matrix.os }}
+
     steps:
       - uses: actions/checkout@v3
       - uses: actions/setup-python@v2
@@ -56,9 +61,9 @@ jobs:
       - name: Run poetry image
         uses: abatilo/actions-poetry@v2.0.0
         with:
-          poetry-version: ${{ matrix.poetry-version }}
+          poetry-version: $POETRY_VERSION
       - name: Install dependencies
-        run: poetry install
+        run: poetry install --with dev
       - name: Run black
         run: poetry run black . --check
       - name: Run isort
@@ -73,3 +78,36 @@ jobs:
         run: poetry run safety check
       - name: Run mypy
         run: poetry run mypy src
+  sphinx-documentation:
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 10
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10"]
+        os: [ubuntu-latest]
+
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Run poetry image
+        uses: abatilo/actions-poetry@v2.0.0
+        with:
+            poetry-version: $POETRY_VERSION
+      - name: Install dependencies
+        run: |
+          poetry install --with dev
+      - name: Build documentation
+        run: |
+          cd docs
+          poetry run make clean
+          poetry run make html
+      - name: Upload documentation as artifacts
+        uses: actions/upload-artifact@v3
+        with:
+          name: docs
+          path: docs/build/html/*
+          retention-days: 5

.pre-commit-config.yaml

@@ -9,7 +9,6 @@ repos:
       - id: mixed-line-ending
       - id: trailing-whitespace
         args: [--markdown-linebreak-ext=md]
-      - id: check-added-large-files
       - id: check-merge-conflict
   - repo: https://github.com/pycqa/isort
     rev: 5.12.0
@@ -29,3 +28,19 @@ repos:
         files: ^\.github/workflows/
         types: [yaml]
         args: ["--schemafile", "https://json.schemastore.org/github-workflow"]
+  - repo: https://github.com/thclark/pre-commit-sphinx
+    rev: 0.0.1
+    hooks:
+      - id: build-docs
+        name: "Check if documentation compiles"
+        args:
+          [
+            "--cache-dir",
+            "docs/build/doctrees",
+            "--html-dir",
+            "docs/build/html",
+            "--source-dir",
+            "docs/source",
+          ]
+        language_version: python3
+        additional_dependencies: [myst-parser]

Changelog.md

@@ -0,0 +1,10 @@
+# Changelog
+
+## 0.2.0
+
+- remove support for Cube until requested
+- add multi database support: Table class accepts any EVAS as `name` parameter without the need to specify the database
+- `setup_credentials` can be used to set up credentials for all three supported databases
+- Improve Table data format: prettify the raw table format and improve readability
+- support new zip format returned by Zensus
+- add proper documentation (using Sphinx)