Skip to content

Commit

Permalink
Full-on switching Breeze development to use uv (#43628)
Browse files Browse the repository at this point in the history
* Full-on switching Breeze development to use uv

Since we are switching to uv, breeze dev environment can now
also be managed with uv and if you fully switch to it, the workflow
of managing breeze, setting the dev environment and upgrading
dependencies is far simpler than airflow's so we should be able
to follow the regular uv workflow for it - i.e. use uv to manage
dependencies, sync, upgrade them etc. This will be really nice
way for some of the maintainers to get to learn uv more and
prepare for more standardisation in the future Airflow development
(maybe) - when we will link uv.lock and constraints management.

Documentation has been updated in Breeze's README to explain some
basic way of using uv for breeze development.
  • Loading branch information
potiuk authored Nov 4, 2024
1 parent 028c14a commit b678857
Show file tree
Hide file tree
Showing 2 changed files with 1,972 additions and 0 deletions.
70 changes: 70 additions & 0 deletions dev/breeze/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [Apache Airflow Breeze](#apache-airflow-breeze)
- [Setting up development env for Breeze](#setting-up-development-env-for-breeze)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

Expand Down Expand Up @@ -62,6 +63,75 @@ available when the package was installed. Since this file becomes part of the in
to detect automatically if any of the files have changed. If they did, the user will be warned to upgrade
their installations.

Setting up development env for Breeze
-------------------------------------

> [!NOTE]
> This section is for developers of Breeze. If you are a user of Breeze, you do not need to read this section.
Breeze is actively developed by Airflow maintainers and contributors, Airflow is an active project
and we are in the process of developing Airflow 3, so breeze requires a lot of adjustments to keep up
the dev environment in sync with Airflow 3 development - this is also why it is part of the same
repository as Airflow - because it needs to be closely synchronized with Airflow development.

As of November 2024 Airflow switches to using `uv` as the main development environment for Airflow
and for Breeze. So the instructions below are for setting up the development environment for Breeze
using `uv`. However we are using only standard python packaging tools, so you can still use `pip` or
`pipenv` or other build frontends to install Breeze, but we recommend using `uv` as it is the most
convenient way to install, manage python packages and virtual environments.

Unlike in Airflow, where we manage our own constraints, we use `uv` to manage requirements for Breeze
and we use `uv` to lock the dependencies. This way we can ensure that the dependencies are always
up-to-date and that the development environment is always consistent for different people. This is
why Breeze's `uv.lock` is committed to the repository and is used to install the dependencies by
default by Breeze. Here's how to install breeze with `uv`


1. Install `uv` - see [uv documentation](https://docs.astral.sh/uv/getting-started/installation/)

> [!IMPORTANT]
> All the commands below should be executed while you are in `dev/breeze` directory of the Airflow repository.
2. Create a new virtual environment for Breeze development:

```shell
uv venv
```

3. Synchronize Breeze dependencies with `uv` to the latest dependencies stored in uv.lock file:

```shell
uv sync
```

After syncing, the `.venv` directory will contain the virtual environment with all the dependencies
installed - you can use that environment to develop Breeze - for example with your favourite IDE
or text editor, you can also use `uv run` to run the scripts in the virtual environment.

For example to run all tests in the virtual environment you can use:

```shell
uv run pytest
```

4. Add/remove dependencies with `uv`:

```shell
uv add <package>
uv remove <package>
```

5. Update and lock the dependencies (after adding them or periodically to keep them up-to-date):

```shell
uv lock
```

Note that when you update dependencies/lock them you should commit the changes in `pyproject.toml` and `uv.lock`.

See [uv documentation](https://docs.astral.sh/uv/getting-started/) for more details on using `uv`.


PLEASE DO NOT MODIFY THE HASH BELOW! IT IS AUTOMATICALLY UPDATED BY PRE-COMMIT.

---------------------------------------------------------------------------------------------------------
Expand Down
Loading

0 comments on commit b678857

Please sign in to comment.