Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ready] Docs changes to remove pandas-iris and update kedro new flow in onboarding docs #3317

Merged
merged 45 commits into from
Nov 27, 2023
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9e9449c
Revise link to notebook docs and remove unnecessary intro page
stichbury Oct 30, 2023
c5b5b0d
Update starters content
stichbury Oct 30, 2023
ce80d99
Merge branch 'develop' into fix-starters-content
stichbury Nov 16, 2023
3cee9f4
relocate starters content
stichbury Nov 16, 2023
124bd90
Added some changes for add-ons and some to do notes
stichbury Nov 16, 2023
b58d9a4
Merge branch 'develop' into fix-starters-content
stichbury Nov 16, 2023
f9062ce
Merge branch 'develop' into fix-starters-content
stichbury Nov 20, 2023
233a4de
Merge branch 'develop' into fix-starters-content
stichbury Nov 21, 2023
4abbdbb
Some further fixes
stichbury Nov 21, 2023
fb5c335
Merge branch 'develop' into fix-starters-content
stichbury Nov 21, 2023
ab9bcd9
Move section about development version of Kedro
stichbury Nov 21, 2023
b787664
Add text for new project
stichbury Nov 21, 2023
47f7e05
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 21, 2023
cd7938e
Remove mention of pandas-iris where possible, replacing with alternative
stichbury Nov 21, 2023
8724441
Merge branch 'develop' into fix-starters-content
stichbury Nov 21, 2023
c4d8ed5
Fix linter errors
stichbury Nov 21, 2023
f7cbffa
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 21, 2023
fc4aed5
Update new project docs
stichbury Nov 22, 2023
2349b7d
Merge branch 'develop' into fix-starters-content
AhdraMeraliQB Nov 22, 2023
bdf10a3
Remove deprecated starters from architecture diagram
stichbury Nov 22, 2023
6e678e6
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 22, 2023
f2a8868
Add warning for pandas-iris usage in generator section
stichbury Nov 22, 2023
50b51c9
Further updates for instances of kedro new
stichbury Nov 22, 2023
5db4cad
Remove TODO as no longer required
Nov 22, 2023
b397ccf
Merge
Nov 22, 2023
d83793e
Resolve some Vale issues and remove implication of tools + starters
stichbury Nov 22, 2023
8708aae
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 22, 2023
edc4e51
fixes to internal links
stichbury Nov 22, 2023
8a832c5
Merge branch 'develop' into fix-starters-content
stichbury Nov 22, 2023
f6e1844
pandas-spaceflights bad, spaceflights-pandas good
stichbury Nov 22, 2023
1c64a78
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 22, 2023
0c66ac3
Merge branch 'develop' into fix-starters-content
stichbury Nov 23, 2023
135ad6a
fix cookiecutter docs urls
stichbury Nov 23, 2023
711bc0b
Update the create a starter docs
stichbury Nov 23, 2023
ccc8f96
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 23, 2023
201776d
Fix link to avoid linkcheck barf
stichbury Nov 23, 2023
b02d672
Update docs/source/get_started/new_project.md
stichbury Nov 23, 2023
57701f6
Update following review
stichbury Nov 23, 2023
aec7ebc
Update content
stichbury Nov 23, 2023
59fdfcf
Merge branch 'fix-starters-content' of https://github.com/kedro-org/k…
stichbury Nov 23, 2023
8c38a11
Update docs/source/nodes_and_pipelines/nodes.md
stichbury Nov 27, 2023
bd04596
Update docs/source/starters/starters.md
stichbury Nov 27, 2023
735ed4f
Update FAQ
stichbury Nov 27, 2023
c09cac5
Merge branch 'develop' into fix-starters-content
stichbury Nov 27, 2023
7cb1dd4
Updates following review
stichbury Nov 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/source/deployment/airflow_astronomer.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The following tutorial uses a different approach and shows how to deploy a Kedro

[Astronomer](https://docs.astronomer.io/astro/install-cli) is a managed Airflow platform which allows users to spin up and run an Airflow cluster easily in production. Additionally, it also provides a set of tools to help users get started with Airflow locally in the easiest way possible.

The tutorial discusses how to run the [example Iris classification pipeline](../get_started/new_project.md#create-a-new-project-containing-example-code) on a local Airflow cluster with Astronomer. You may also consider using our [`astro-airflow-iris` starter](https://github.com/kedro-org/kedro-starters/tree/main/astro-airflow-iris) which provides a template containing the boilerplate code that the tutorial describes:
The tutorial discusses how to run the example Iris classification pipeline on a local Airflow cluster with Astronomer. You may also consider using our [`astro-airflow-iris` starter](https://github.com/kedro-org/kedro-starters/tree/main/astro-airflow-iris) which provides a template containing the boilerplate code that the tutorial describes:

```shell
kedro new --starter=astro-airflow-iris
Expand Down Expand Up @@ -44,10 +44,10 @@ To follow this tutorial, ensure you have the following:
astro dev init
```

2. Create a new Kedro project using the `pandas-iris` starter. You can use the default value in the project creation process:
2. Create a new Kedro project using the `astro-airflow-iris` starter. You can use the default value in the project creation process:

```shell
kedro new --starter=pandas-iris
kedro new --starter=astro-airflow-iris
```

3. Copy all files and directories under `new-kedro-project`, which was the default project name created in step 2, to the root directory so Kedro and Astro CLI share the same project root:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/deployment/distributed.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@

## 4. (Optional) Create starters

This is an optional step, but it may speed up your work in the long term. If you find yourself having to deploy in a similar environment or to a similar platform fairly often, you may want to [build your own Kedro starter](../kedro_project_setup/starters.md). That way you will be able to re-use any deployment scripts written as part of step 2.
You may opt to [build your own Kedro starter](../starters/starters.md) if you regularly have to deploy in a similar environment or to a similar platform. The starter enables you to re-use any deployment scripts written as part of step 2.

Check warning on line 43 in docs/source/deployment/distributed.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/deployment/distributed.md#L43

[Kedro.weaselwords] 'regularly' is a weasel word!
Raw output
{"message": "[Kedro.weaselwords] 'regularly' is a weasel word!", "location": {"path": "docs/source/deployment/distributed.md", "range": {"start": {"line": 43, "column": 79}}}, "severity": "WARNING"}
2 changes: 1 addition & 1 deletion docs/source/extend_kedro/architecture_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Kedro framework serves as the interface between a Kedro project and Kedro librar

## Kedro starter

You can use a [Kedro starter](../kedro_project_setup/starters.md) to generate a Kedro project that contains boilerplate code. We maintain a set of [official starters](https://github.com/kedro-org/kedro-starters/) but you can also use a custom starter of your choice.
You can use a [Kedro starter](../starters/starters.md) to generate a Kedro project that contains boilerplate code. We maintain a set of [official starters](https://github.com/kedro-org/kedro-starters/) but you can also use a custom starter of your choice.

## Kedro library

Expand Down
2 changes: 1 addition & 1 deletion docs/source/extend_kedro/common_use_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ Your plugin's implementation can take advantage of other extension mechanisms su

## Use Case 4: How to customise the initial boilerplate of your project

Sometimes you might want to tailor the starting boilerplate of a Kedro project to your specific needs. For example, your organisation might have a standard CI script that you want to include in every new Kedro project. To this end, see the [guide for creating Kedro starters](../kedro_project_setup/starters.md#how-to-create-a-kedro-starter).
Sometimes you might want to tailor the starting boilerplate of a Kedro project to your specific needs. For example, your organisation might have a standard CI script that you want to include in every new Kedro project. To this end, see the [guide for creating Kedro starters](../starters/create_a_starter.md).
1 change: 1 addition & 0 deletions docs/source/extend_kedro/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
common_use_cases
plugins
architecture_overview
../starters/create_a_starter
stichbury marked this conversation as resolved.
Show resolved Hide resolved
```
43 changes: 0 additions & 43 deletions docs/source/extend_kedro/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,49 +42,6 @@ Once the plugin is installed, you can run it as follows:
kedro to_json
```

## Extend starter aliases
It is possible to extend the list of starter aliases built into Kedro. This means that a [custom Kedro starter](../kedro_project_setup/starters.md#how-to-create-a-kedro-starter) can be used directly through the `starter` argument in `kedro new` rather than needing to explicitly provide the `template` and `directory` arguments. A custom starter alias behaves in the same way as an official Kedro starter alias and is also picked up by `kedro starter list`.

You need to extend the starters by providing a list of `KedroStarterSpec`, in this example it is defined in a file called `plugin.py`.

Example for a non-git repository starter:
```python
# plugin.py
starters = [
KedroStarterSpec(
alias="test_plugin_starter",
template_path="your_local_directory/starter_folder",
)
]
```

Example for a git repository starter:
```python
# plugin.py
starters = [
KedroStarterSpec(
alias="test_plugin_starter",
template_path="https://github.com/kedro-org/kedro-starters/",
directory="spaceflights-pandas",
)
]
```

The `directory` argument is optional and should be used when you have multiple templates in one repository as for the [official kedro-starters](https://github.com/kedro-org/kedro-starters). If you only have one template, your top-level directory will be treated as the template. For an example, see the [spaceflights-pandas starter](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas).

In your `pyproject.toml`, you need to register the specifications to `kedro.starters`:

```toml
[project.entry-points."kedro.starters"]
starter = "plugin:starters"
```

After that you can use this starter with `kedro new --starter=test_plugin_starter`.

```{note}
If your starter lives on a git repository, by default Kedro attempts to use a tag or branch labelled with your version of Kedro, e.g. `0.18.12`. This means that you can host different versions of your starter template on the same repository, and the correct one will automatically be used. If you do not wish to follow this structure, you should override it with the `checkout` flag, e.g. `kedro new --starter=test_plugin_starter --checkout=main`.
```

## Working with `click`

Commands must be provided as [`click` `Groups`](https://click.palletsprojects.com/en/7.x/api/#click.Group)
Expand Down
29 changes: 28 additions & 1 deletion docs/source/faq/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

## Kedro project development

* [How do I write my own Kedro starter projects](../kedro_project_setup/starters.md#how-to-create-a-kedro-starter)?
* [How do I write my own Kedro starter projects](../starters/create_a_starter.md)?

Check warning on line 16 in docs/source/faq/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/faq/faq.md#L16

[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.
Raw output
{"message": "[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.", "location": {"path": "docs/source/faq/faq.md", "range": {"start": {"line": 16, "column": 11}}}, "severity": "WARNING"}

Check warning on line 16 in docs/source/faq/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/faq/faq.md#L16

[Kedro.pronouns] Avoid first-person singular pronouns such as 'my'.
Raw output
{"message": "[Kedro.pronouns] Avoid first-person singular pronouns such as 'my'.", "location": {"path": "docs/source/faq/faq.md", "range": {"start": {"line": 16, "column": 19}}}, "severity": "WARNING"}

## Configuration

Expand Down Expand Up @@ -67,3 +67,30 @@
| Models | Stored, serialised pre-trained machine learning models |
| Model output | Analytics specific data model(s) containing the results generated by the model based on the `model input` data |
| Reporting | Reporting data model(s) that are used to combine a set of `primary`, `feature`, `model input` and `model output` data used to drive the dashboard and the views constructed. It encapsulates and removes the need to define any blending or joining of data, improve performance and replacement of presentation layer without having to redefine the data models |

## How can I check the version of Kedro installed?

Check warning on line 71 in docs/source/faq/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/faq/faq.md#L71

[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.
Raw output
{"message": "[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.", "location": {"path": "docs/source/faq/faq.md", "range": {"start": {"line": 71, "column": 12}}}, "severity": "WARNING"}
To check the version installed, type `kedro -V` in your terminal window.


## How do I install a development version of Kedro?

Check warning on line 75 in docs/source/faq/faq.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/source/faq/faq.md#L75

[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.
Raw output
{"message": "[Kedro.pronouns] Avoid first-person singular pronouns such as 'I'.", "location": {"path": "docs/source/faq/faq.md", "range": {"start": {"line": 75, "column": 11}}}, "severity": "WARNING"}

This section explains how to try out a development version of Kedro direct from the [Kedro GitHub repository](https://github.com/kedro-org/kedro).
stichbury marked this conversation as resolved.
Show resolved Hide resolved

```{important}
The development version of Kedro is not guaranteed to be bug-free and/or compatible with any of the [stable versions](https://pypi.org/project/kedro/#history). We do not recommend that you use a development version of Kedro in any production systems. Please install and use with caution.
```

To try out latest, unreleased functionality from the `develop` branch of the Kedro GitHub repository, run the following installation command:

```bash
pip install git+https://github.com/kedro-org/kedro.git@develop
```

This will install Kedro from the `develop` branch of the GitHub repository, which is always the most up to date. This command will install Kedro from source, unlike `pip install kedro` which installs Kedro from PyPI.

If you want to roll back to a stable version of Kedro, execute the following in your environment:

```bash
pip uninstall kedro -y
pip install kedro
```
22 changes: 0 additions & 22 deletions docs/source/get_started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,28 +162,6 @@ When migrating an existing project to a newer Kedro version, make sure you also
* For projects generated with versions of Kedro > 0.17.0, you'll do this in the `pyproject.toml` file from the project root directory.
* If your project was generated with a version of Kedro <0.17.0, you will instead need to update the `ProjectContext`, which is found in `src/<package_name>/run.py`.

## How to install a development version of Kedro

This section explains how to try out a development version of Kedro direct from the [Kedro GitHub repository](https://github.com/kedro-org/kedro).

```{important}
The development version of Kedro is not guaranteed to be bug-free and/or compatible with any of the [stable versions](https://pypi.org/project/kedro/#history). We do not recommend that you use a development version of Kedro in any production systems. Please install and use with caution.
```

To try out latest, unreleased functionality from the `develop` branch of the Kedro GitHub repository, run the following installation command:

```bash
pip install git+https://github.com/kedro-org/kedro.git@develop
```

This will install Kedro from the `develop` branch of the GitHub repository, which is always the most up to date. This command will install Kedro from source, unlike `pip install kedro` which installs Kedro from PyPI.

If you want to roll back to a stable version of Kedro, execute the following in your environment:

```bash
pip uninstall kedro -y
pip install kedro
```

## Summary

Expand Down
Loading