Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove renamed/archived starters #3322

Merged
merged 5 commits into from
Nov 21, 2023
Merged

Remove renamed/archived starters #3322

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0ee1704
Remove renamed/archived starters
merelcht Nov 20, 2023
a8cca8e
Fix test
merelcht Nov 20, 2023
1fff64e
Replace pandas-iris by spaceflights-pandas for simple cases in the docs
merelcht Nov 20, 2023
becf9d5
Update .gitpod.yml
merelcht Nov 20, 2023
fc827c5
Merge branch 'develop' into remove-archived-starters
merelcht Nov 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitpod.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tasks:
make sign-off
pip install -e /workspace/kedro[test]
cd /workspace
yes project | kedro new -s pandas-iris --checkout main
kedro new --name project -s spaceflights-pandas --checkout main
cd /workspace/kedro
pre-commit install --install-hooks

Expand Down
4 changes: 2 additions & 2 deletions docs/source/extend_kedro/plugins.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,12 +65,12 @@
KedroStarterSpec(
alias="test_plugin_starter",
template_path="https://github.com/kedro-org/kedro-starters/",
directory="pandas-iris",
directory="spaceflights-pandas",
)
]
```

The `directory` argument is optional and should be used when you have multiple templates in one repository as for the [official kedro-starters](https://github.com/kedro-org/kedro-starters). If you only have one template, your top-level directory will be treated as the template. For an example, see the [pandas-iris starter](https://github.com/kedro-org/kedro-starters/tree/main/pandas-iris).
The `directory` argument is optional and should be used when you have multiple templates in one repository as for the [official kedro-starters](https://github.com/kedro-org/kedro-starters). If you only have one template, your top-level directory will be treated as the template. For an example, see the [spaceflights-pandas starter](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas).

Check warning on line 73 in docs/source/extend_kedro/plugins.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/extend_kedro/plugins.md#L73 

[Kedro.toowordy] 'multiple' is too wordy
Raw output 
{"message": "[Kedro.toowordy] 'multiple' is too wordy", "location": {"path": "docs/source/extend_kedro/plugins.md", "range": {"start": {"line": 73, "column": 71}}}, "severity": "WARNING"}

Check warning on line 73 in docs/source/extend_kedro/plugins.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/extend_kedro/plugins.md#L73 

[Kedro.weaselwords] 'only' is a weasel word!
Raw output 
{"message": "[Kedro.weaselwords] 'only' is a weasel word!", "location": {"path": "docs/source/extend_kedro/plugins.md", "range": {"start": {"line": 73, "column": 198}}}, "severity": "WARNING"}

In your `pyproject.toml`, you need to register the specifications to `kedro.starters`:

Expand Down
4 changes: 0 additions & 4 deletions docs/source/kedro_project_setup/starters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,6 @@ kedro starter list
The Kedro team maintains the following starters for a range of Kedro projects:

* [`astro-airflow-iris`](https://github.com/kedro-org/kedro-starters/tree/main/astro-airflow-iris): The [Kedro Iris dataset example project](../get_started/new_project.md) with a minimal setup for deploying the pipeline on Airflow with [Astronomer](https://www.astronomer.io/).
* [`standalone-datacatalog`](https://github.com/kedro-org/kedro-starters/tree/main/standalone-datacatalog): A minimum setup to use the traditional [Iris dataset](https://www.kaggle.com/uciml/iris) with Kedro's [`DataCatalog`](../data/data_catalog.md), which is a core component of Kedro. This starter is of use in the exploratory phase of a project. It was formerly known as `mini-kedro`.
* [`pandas-iris`](https://github.com/kedro-org/kedro-starters/tree/main/pandas-iris): The [Kedro Iris dataset example project](../get_started/new_project.md)
* [`pyspark-iris`](https://github.com/kedro-org/kedro-starters/tree/main/pyspark-iris): An alternative Kedro Iris dataset example, using [PySpark](../integrations/pyspark_integration.md)
* [`pyspark`](https://github.com/kedro-org/kedro-starters/tree/main/pyspark): The configuration and initialisation code for a [Kedro pipeline using PySpark](../integrations/pyspark_integration.md)
* [`spaceflights-pandas`](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas): The [spaceflights tutorial](../tutorial/spaceflights_tutorial.md) example code with `pandas` datasets.
* [`spaceflights-pandas-viz`](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas-viz): The [spaceflights tutorial](../tutorial/spaceflights_tutorial.md) example code with `pandas` datasets and visualisation and experiment tracking `kedro-viz` features.
* [`spaceflights-pyspark`](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pyspark): The [spaceflights tutorial](../tutorial/spaceflights_tutorial.md) example code with `pyspark` datasets.
Expand Down
25 changes: 0 additions & 25 deletions kedro/framework/cli/starters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,26 +87,9 @@ class KedroStarterSpec: # noqa: too-few-public-methods
KEDRO_PATH = Path(kedro.__file__).parent
TEMPLATE_PATH = KEDRO_PATH / "templates" / "project"

_DEPRECATED_STARTERS = [
"pandas-iris",
"pyspark-iris",
"pyspark",
"standalone-datacatalog",
]
_STARTERS_REPO = "git+https://github.com/kedro-org/kedro-starters.git"
_OFFICIAL_STARTER_SPECS = [
KedroStarterSpec("astro-airflow-iris", _STARTERS_REPO, "astro-airflow-iris"),
# The `astro-iris` was renamed to `astro-airflow-iris`, but old (external)
# documentation and tutorials still refer to `astro-iris`. We create an alias to
# check if a user has entered old `astro-iris` as the starter name and changes it
# to `astro-airflow-iris`.
KedroStarterSpec("astro-iris", _STARTERS_REPO, "astro-airflow-iris"),
KedroStarterSpec(
"standalone-datacatalog", _STARTERS_REPO, "standalone-datacatalog"
),
KedroStarterSpec("pandas-iris", _STARTERS_REPO, "pandas-iris"),
KedroStarterSpec("pyspark", _STARTERS_REPO, "pyspark"),
KedroStarterSpec("pyspark-iris", _STARTERS_REPO, "pyspark-iris"),
KedroStarterSpec("spaceflights-pandas", _STARTERS_REPO, "spaceflights-pandas"),
KedroStarterSpec(
"spaceflights-pandas-viz", _STARTERS_REPO, "spaceflights-pandas-viz"
Expand All @@ -123,7 +106,6 @@ class KedroStarterSpec: # noqa: too-few-public-methods

_OFFICIAL_STARTER_SPECS = {spec.alias: spec for spec in _OFFICIAL_STARTER_SPECS}


ADD_ONS_SHORTNAME_TO_NUMBER = {
"lint": "1",
"test": "2",
Expand Down Expand Up @@ -361,13 +343,6 @@ def _get_starters_dict() -> dict[str, KedroStarterSpec]:
directory="astro-airflow-iris",
origin="kedro"
),
"astro-iris":
KedroStarterSpec(
name="astro-iris",
template_path="git+https://github.com/kedro-org/kedro-starters.git",
directory="astro-airflow-iris",
origin="kedro"
),
}
"""
starter_specs = _OFFICIAL_STARTER_SPECS
Expand Down
2 changes: 1 addition & 1 deletion tests/framework/cli/test_starters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -866,7 +866,7 @@ def test_directory_flag_without_starter(self, fake_kedro_cli):
def test_directory_flag_with_starter_alias(self, fake_kedro_cli):
result = CliRunner().invoke(
fake_kedro_cli,
["new", "--starter", "pyspark-iris", "--directory", "some-dir"],
["new", "--starter", "spaceflights-pandas", "--directory", "some-dir"],
input=_make_cli_prompt_input(),
)
assert result.exit_code != 0
Expand Down