Move default template to static pyproject.toml, take 2 (#2853)

* Remove leftover reference to micropkg setup.py Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Replace outdated extras requirement with kedro-datasets Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Remove outdated mention to environment.yml Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Move flake8 configuration to dedicated file Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Replace setup.py with pyproject.toml in default project template Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Adapt link to moving file in docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Replace outdated extras requirement with kedro-datasets" This reverts commit dfa8a9e. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Add release notes for pyproject.toml change Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
kedro-org · Aug 25, 2023 · e86508e · e86508e
1 parent e2bb4cd
commit e86508e
Show file tree

Hide file tree

Showing 15 changed files with 78 additions and 92 deletions.
diff --git a/RELEASE.md b/RELEASE.md
@@ -19,6 +19,8 @@
 
 ## Bug fixes and other changes
 * Updated `kedro pipeline create` and `kedro catalog create` to use new `/conf` file structure.
+* Converted `setup.py` in default template to `pyproject.toml` and moved flake8 configuration
+  to dedicated file `.flake8`.
 
 ## Documentation changes
 * Revised the `data` section to restructure beginner and advanced pages about the Data Catalog and datasets.

diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -227,7 +227,9 @@
     "https://opensource.org/license/apache2-0-php/",
     "https://docs.github.com/en/rest/overview/other-authentication-methods#via-username-and-password",
     "https://docs.snowflake.com/en/developer-guide/snowpark/reference/python/api/snowflake.snowpark.DataFrameWriter.saveAsTable.html",
-    "https://www.educative.io/blog/advanced-yaml-syntax-cheatsheet#anchors"
+    "https://www.educative.io/blog/advanced-yaml-syntax-cheatsheet#anchors",
+    # temporarily, until the file actually exists in `main`
+    "https://github.com/kedro-org/kedro/blob/main/kedro/templates/project/%7B%7B%20cookiecutter.repo_name%20%7D%7D/.flake8",
 ]
 
 # retry before render a link broken (fix for "too many requests")

diff --git a/docs/source/deployment/airflow_astronomer.md b/docs/source/deployment/airflow_astronomer.md
@@ -74,7 +74,7 @@ To follow this tutorial, ensure you have the following:
     ├── plugins
     ├── pyproject.toml
     ├── requirements.txt
-    ├── setup.cfg
+    ├── .flake8
     └── src
     ```
 

diff --git a/docs/source/development/linting.md b/docs/source/development/linting.md
@@ -60,7 +60,7 @@ It is a good practice to [split your line when it is too long](https://beta.ruff
 
 #### Configure `flake8`
 
-Store your `flake8` configuration in a file named `setup.cfg` within your project root. The Kedro starters use the [following configuration](https://github.com/kedro-org/kedro-starters/blob/main/pandas-iris/%7B%7B%20cookiecutter.repo_name%20%7D%7D/setup.cfg):
+Store your `flake8` configuration in a file named `.flake8` within your project root. The Kedro default project template use the [following configuration](https://github.com/kedro-org/kedro/blob/main/kedro/templates/project/%7B%7B%20cookiecutter.repo_name%20%7D%7D/.flake8):
 
 ```text
 [flake8]

diff --git a/docs/source/get_started/kedro_concepts.md b/docs/source/get_started/kedro_concepts.md
@@ -72,7 +72,7 @@ project-dir         # Parent directory of the template
 ├── notebooks       # Project-related Jupyter notebooks (can be used for experimental code before moving the code to src)
 ├── pyproject.toml  # Identifies the project root and contains configuration information
 ├── README.md       # Project README
-├── setup.cfg       # Configuration options for `pytest` when doing `kedro test` and for the `isort` utility when doing `kedro lint`
+├── .flake8         # Configuration options for `flake8` (linting)
 └── src             # Project source code
 ```
 

diff --git a/docs/source/kedro_project_setup/starters.md b/docs/source/kedro_project_setup/starters.md
@@ -155,7 +155,7 @@ Here is the layout of the project as a Cookiecutter template:
 ├── docs                         # Project documentation
 ├── notebooks                    # Project related Jupyter notebooks (can be used for experimental code before moving the code to src)
 ├── README.md                    # Project README
-├── setup.cfg                    # Configuration options for tools e.g. `pytest` or `black`
+├── .flake8                      # Configuration options for `flake8` (linting)
 └── src                          # Project source code
     └── {{ cookiecutter.python_package }}
        ├── __init.py__
@@ -164,7 +164,7 @@ Here is the layout of the project as a Cookiecutter template:
        ├── __main__.py
        └── settings.py
     ├── requirements.txt
-    ├── setup.py
+    ├── pyproject.toml
     └── tests
 ```
 

diff --git a/features/steps/test_starter/{{ cookiecutter.repo_name }}/.flake8 b/features/steps/test_starter/{{ cookiecutter.repo_name }}/.flake8
@@ -0,0 +1,3 @@
+[flake8]
+max-line-length = 88
+extend-ignore = E203
diff --git a/features/steps/test_starter/{{ cookiecutter.repo_name }}/README.md b/features/steps/test_starter/{{ cookiecutter.repo_name }}/README.md
@@ -17,7 +17,7 @@ In order to get the best out of the template:
 
 ## How to install dependencies
 
-Declare any dependencies in `src/requirements.txt` for `pip` installation and `src/environment.yml` for `conda` installation.
+Declare any dependencies in `src/requirements.txt` for `pip` installation.
 
 To install them, run:
 

diff --git a/features/steps/test_starter/{{ cookiecutter.repo_name }}/setup.cfg b/features/steps/test_starter/{{ cookiecutter.repo_name }}/setup.cfg
diff --git a/features/steps/test_starter/{{ cookiecutter.repo_name }}/src/pyproject.toml b/features/steps/test_starter/{{ cookiecutter.repo_name }}/src/pyproject.toml
@@ -0,0 +1,31 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "{{ cookiecutter.python_package }}"
+version = "0.1"
+dynamic = ["dependencies"]
+
+[tool.setuptools.packages.find]
+exclude = ["tests"]
+
+[project.scripts]
+{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"
+
+[project.optional-dependencies]
+docs = [
+    "docutils<0.18.0",
+    "sphinx~=3.4.3",
+    "sphinx_rtd_theme==0.5.1",
+    "nbsphinx==0.8.1",
+    "nbstripout~=0.4",
+    "sphinx-autodoc-typehints==1.11.1",
+    "sphinx_copybutton==0.3.1",
+    "ipykernel>=5.3, <7.0",
+    "Jinja2<3.1.0",
+    "myst-parser~=0.17.2",
+]
+
+[tool.setuptools.dynamic]
+dependencies = {file = "requirements.txt"}
diff --git a/features/steps/test_starter/{{ cookiecutter.repo_name }}/src/setup.py b/features/steps/test_starter/{{ cookiecutter.repo_name }}/src/setup.py
diff --git a/...ct/{{ cookiecutter.repo_name }}/setup.cfg → ...ject/{{ cookiecutter.repo_name }}/.flake8 b/...ct/{{ cookiecutter.repo_name }}/setup.cfg → ...ject/{{ cookiecutter.repo_name }}/.flake8
diff --git a/kedro/templates/project/{{ cookiecutter.repo_name }}/src/pyproject.toml b/kedro/templates/project/{{ cookiecutter.repo_name }}/src/pyproject.toml
@@ -0,0 +1,31 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "{{ cookiecutter.python_package }}"
+version = "0.1"
+dynamic = ["dependencies"]
+
+[tool.setuptools.packages.find]
+exclude = ["tests"]
+
+[project.scripts]
+{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"
+
+[project.optional-dependencies]
+docs = [
+    "docutils<0.18.0",
+    "sphinx~=3.4.3",
+    "sphinx_rtd_theme==0.5.1",
+    "nbsphinx==0.8.1",
+    "nbstripout~=0.4",
+    "sphinx-autodoc-typehints==1.11.1",
+    "sphinx_copybutton==0.3.1",
+    "ipykernel>=5.3, <7.0",
+    "Jinja2<3.1.0",
+    "myst-parser~=0.17.2",
+]
+
+[tool.setuptools.dynamic]
+dependencies = {file = "requirements.txt"}
diff --git a/kedro/templates/project/{{ cookiecutter.repo_name }}/src/setup.py b/kedro/templates/project/{{ cookiecutter.repo_name }}/src/setup.py
diff --git a/tests/framework/cli/micropkg/test_micropkg_requirements.py b/tests/framework/cli/micropkg/test_micropkg_requirements.py
@@ -249,8 +249,8 @@ def test_empty_pipeline_requirements_txt(
     def test_complex_requirements(
         self, requirement, fake_project_cli, fake_metadata, fake_package_path
     ):
-        """Options that are valid in requirements.txt but cannot be packaged using
-        setup.py."""
+        """Options that are valid in requirements.txt but cannot be packaged in
+        pyproject.toml."""
         self.call_pipeline_create(fake_project_cli, fake_metadata)
         pipeline_requirements_txt = (
             fake_package_path / "pipelines" / PIPELINE_NAME / "requirements.txt"