Skip to content

Commit

Permalink
added some documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
mbway committed Feb 29, 2024
1 parent a083083 commit 2cda00f
Show file tree
Hide file tree
Showing 7 changed files with 184 additions and 295 deletions.
28 changes: 28 additions & 0 deletions Contributing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Contributing
Thank you for your interest in contributing to maturin_import_hook. All are welcome! Please consider reading
the [Code of Conduct](https://github.com/PyO3/maturin-import-hook/blob/main/Code-of-Conduct.md) to keep our community
positive and inclusive.

## Getting Started Contributing
- we use the [github issue tracker](https://github.com/PyO3/maturin-import-hook/issues) and [discussions](https://github.com/PyO3/maturin-import-hook/discussions) to keep track of bugs and ideas

### Setting up a development environment

1. Install rust (eg using `rustup`) and maturin (eg `pipx install maturin`)
2. Clone the repository
- if you are looking to submit a PR, create a fork on github and clone your fork
3. Install [pre-commit](https://pre-commit.com/) (eg `pipx install pre-commit`) then run `pre-commit install` in the repo root
4. See [tests/README.md](https://github.com/PyO3/maturin-import-hook/blob/main/tests/README.md) for instructions on how best to run the test suite and some other miscellaneous instructions for debugging and carrying out maintenance tasks.

Tips:
- [pyenv](https://github.com/pyenv/pyenv) may be useful for installing a specific python interpreter
- Virtual machines such as [VirtualBox](https://www.virtualbox.org/) and [OSX-KVM](https://github.com/kholia/OSX-KVM) are useful for running tests on specific platforms locally. Or you can use the test pipeline on github actions.

## Writing Pull Requests

### Continuous Integration
The maturin_import_hook repo uses [GitHub actions](https://github.com/PyO3/maturin-import-hook/actions). PRs are blocked from merging if the CI is not successful.

You can run the test pipeline on your fork from the 'Actions' tab of your fork. The pipeline takes several arguments when run manually that you can use to narrow down what is run so you can iterate faster when trying to solve a particular issue. The pipeline uploads the test results as html reports that can be downloaded and examined. This is generally easier than sifting through the raw logs.

Linting and type-checking is enforced in the repo using [pre-commit](https://pre-commit.com/). See `.pre-commit-config.yaml` for the checks that are performed and `pyproject.toml` for the configuration of those linters. The configuration starts with all `ruff` lints enabled with a list of specifically disabled lints. If you are writing new code that is triggering a lint that you think ought to be disabled, you can suggest this in a PR, but generally stick to conforming to the suggested linter rules.
21 changes: 16 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,29 @@ After installing `maturin`, install the import hook into a python virtual enviro
```shell
$ pip install maturin_import_hook
```

Then run the following command to install the import hook into [sitecustomize.py](https://docs.python.org/3/library/site.html#module-sitecustomize)
so that it activates automatically:
Then, optionally make the hook activate automatically with:
```shell
$ python -m maturin_import_hook site install
```
This only has to be run once for each virtual environment. Uninstall with `python -m maturin_import_hook site uninstall`
This only has to be run once for each virtual environment.

Or alternatively, put the following at the top of each python script where you want the import hook to be active:
Alternatively, put the following at the top of each python script where you want to use the hook:
```python
import maturin_import_hook

# install the import hook with default settings.
# this call must be before any imports that you want the hook to be active for.
maturin_import_hook.install()

# when a rust package that is installed in editable mode is imported,
# that package will be automatically recompiled if necessary.
import pyo3_pure

# when a .rs file is imported a project will be created for it in the
# maturin build cache and the resulting library will be loaded.
#
# assuming subpackage/my_rust_script.rs defines a pyo3 module:
import subpackage.my_rust_script
```

Once the hook is active, any `import` statement that imports an editable-installed maturin project will be
Expand Down
20 changes: 20 additions & 0 deletions docs/advanced.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Advanced Usage

The import hook classes can be subclassed to further customize to specific use cases.
For example settings can be configured per-project or loaded from configuration files.
```python
import sys
from pathlib import Path
from maturin_import_hook.settings import MaturinSettings
from maturin_import_hook.project_importer import MaturinProjectImporter

class CustomImporter(MaturinProjectImporter):
def get_settings(self, module_path: str, source_path: Path) -> MaturinSettings:
return MaturinSettings(
release=True,
strip=True,
# ...
)

sys.meta_path.insert(0, CustomImporter())
```
113 changes: 113 additions & 0 deletions docs/basics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Basics

`maturin_import_hook` is a package that provides the capability for python `import` statements to trigger a rebuild
when importing a maturin project that is installed in editable mode (eg with `maturin develop` or `pip install -e`).
This makes development much more convenient as it brings the workflow of
developing Rust modules closer to the workflow of developing regular python modules.

The hook supports importing editable-installed pure Rust and mixed Rust/Python project
layouts as well as importing standalone `.rs` files.

## Installation

Install into a virtual environment then install so that the import hook is always active.
```shell
$ pip install maturin_import_hook
$ python -m maturin_import_hook site install # install into the active environment
```

Alternatively, instead of using `site install`, put calls to `maturin_import_hook.install()` into any script where you
want to use the import hook.

## Usage

```python
import maturin_import_hook

# install the import hook with default settings.
# this must be called before importing any maturin project
maturin_import_hook.install()

# when a rust package that is installed in editable mode is imported,
# that package will be automatically recompiled if necessary.
import pyo3_pure

# when a .rs file is imported a project will be created for it in the
# maturin build cache and the resulting library will be loaded.
#
# assuming subpackage/my_rust_script.rs defines a pyo3 module:
import subpackage.my_rust_script
```

The maturin project importer and the rust file importer can be used separately
```python
from maturin_import_hook import rust_file_importer
rust_file_importer.install()

from maturin_import_hook import project_importer
project_importer.install()
```

The import hook can be configured to control its behaviour
```python
import maturin_import_hook
from maturin_import_hook.settings import MaturinSettings

maturin_import_hook.install(
enable_project_importer=True,
enable_rs_file_importer=True,
settings=MaturinSettings(
release=True,
strip=True,
# ...
),
show_warnings=True,
# ...
)
```

Since the import hook is intended for use in development environments and not for
production environments, it may be a good idea to put the call to `maturin_import_hook.install()`
into `site-packages/sitecustomize.py` of your development virtual environment
([documentation](https://docs.python.org/3/library/site.html)). This will
enable the hook for every script run by that interpreter without calling `maturin_import_hook.install()`
in every script, meaning the scripts do not need alteration before deployment.

Installation into `sitecustomize.py` can be managed with the import hook cli using
`python -m maturin_import_hook site install`. The CLI can also manage uninstallation.

## Environment Variables
The import hook can be disabled by setting `MATURIN_IMPORT_HOOK_ENABLED=0`. This can be used to disable
the import hook in production if you want to leave calls to `import_hook.install()` in place.

Build files will be stored in an appropriate place for the current system but can be overridden
by setting `MATURIN_BUILD_DIR`. These files can be deleted without causing any issues (unless a build is in progress).
The precedence for storing build files is:

* `MATURIN_BUILD_DIR`
* `<virtualenv_dir>/maturin_build_cache`
* `<system_cache_dir>/maturin_build_cache`
* e.g. `~/.cache/maturin_build_cache` on POSIX

See the location being used with the CLI: `python -m maturin_import_hook cache info`


## Logging
By default the `maturin_import_hook` logger does not propagate to the root logger. This is so that `INFO` level messages
are shown to the user without them having to configure logging (`INFO` level is normally not visible). The import hook
also has extensive `DEBUG` level logging that generally would be more noise than useful. So by not propagating, `DEBUG`
messages from the import hook are not shown even if the root logger has `DEBUG` level visible.

If you prefer, `maturin_import_hook.reset_logger()` can be called to undo the default configuration and propagate
the messages as normal.

When debugging issues with the import hook, you should first call `reset_logger()` then configure the root logger
to show `DEBUG` messages. You can also run with the environment variable `RUST_LOG=maturin=debug` to get more
information from maturin.
```python
import logging
logging.basicConfig(format='%(name)s [%(levelname)s] %(message)s', level=logging.DEBUG)
import maturin_import_hook
maturin_import_hook.reset_logger()
maturin_import_hook.install()
```
7 changes: 7 additions & 0 deletions docs/reloading.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Reloading

This document outlines the implementation details of how the import hook supports `importlib.reload()`. From a user's
perspective the important thing is that `importlib.reload()` should do what you expect (with some caveats) when the
import hook is active


# Reloading Details

Regular python modules can be reloaded with [`importlib.reload()`](https://docs.python.org/3/library/importlib.html#importlib.reload)
to load changes to the source code without restarting the python interpreter.
This mechanism is used by the [`%autoreload` IPython extension](https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html)
Expand Down
40 changes: 0 additions & 40 deletions guide/src/environment-variables.md

This file was deleted.

Loading

0 comments on commit 2cda00f

Please sign in to comment.