A pre-commit plugin to align pre-commit repository versions with those derived by pip-tools.
pip-tools and Pre-Commit are two of my favourite development tools. However, they don't always necessarily agree on what versions of packages should be installed. This lack of consolidation can lead to problems with pre-commit. This pre-commit plugin syncs the pre-commit package versions with the versions generated by pip-tools' compile process.
A while ago I came across an issue committing files to my local git repository. The issue occured at the flake8 linting stage using pre-commit. There was however no problem manually running flake8 from the command line. Upon investigation It was found that flake8 had an incompatibility issue with one of its plugins. With further investigation I noticed that pip-tools had pinned flake8 to an earlier version to what pre-commit was caching.
In short - pip-tools does a spectacularly good job pinning your dependencies and the dependencies of these dependencies. It has one job to do and it does it perfectly. The pre-commit autoupdate command just updates the "rev" for the "repo" in the .pre-commit-config.yaml file to the latest version available and ignores the dependencies.
The following example displays the way flake8 and a few plugins are configured by the two tools :
- pip-tools (via requirements.txt)
flake8==4.0.1
# via ...
flake8-bugbear==22.8.23
# via ...
flake8-comprehensions==3.10.0
# via ...
flake8-eradicate==1.3.0
# via ...
flake8-simplify==0.19.3
# via ...- pre-commit (via .pre-commit-config.yaml)
- repo: https://github.com/pycqa/flake8
rev 5.0.4
hooks:
- id: flake8
additional_dependencies:
- flake8-bugbear
- flake8-comprehensions
- flake8-eradicate
- flake8-simplify
args: ["--ignore=E800,F842,F841,W503"]If we force pip-tools to use a later version of flake8 (e,g 5.0.4) by manually pinning the version in the ".in" file and recompiling it is clear pip-tools is unhappy... A quick look at the dependencies shows why.
Could not find a version that matches flake8!=3.2.0,<5.0.0,<6,==5.0.4,>=3.0.0,>=3.3.0,>=3.5,>=3.7,>=3.9.1,>=4.0.1 (from -r requirements\development.in (line 12))
Tried: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.3.1, 1.4, 1.5, 1.6, 1.6.1, 1.6.2, 1.7.0, 2.0, 2.1.0, 2.2.0, 2.2.0, 2.2.1, 2.2.1, 2.2.2, 2.2.2, 2.2.3, 2.2.3, 2.2.4, 2.2.5, 2.3.0, 2.3.0, 2.4.0, 2.4.0, 2.4.1, 2.4.1, 2.5.0, 2.5.0, 2.5.1, 2.5.1, 2.5.2, 2.5.2, 2.5.3, 2.5.3, 2.5.4, 2.5.4, 2.5.5, 2.5.5, 2.6.0, 2.6.0, 2.6.1, 2.6.1, 2.6.2, 2.6.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.3.0, 3.3.0, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.5.0, 3.5.0, 3.6.0, 3.6.0, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.7.5, 3.7.5, 3.7.6, 3.7.6, 3.7.7, 3.7.7, 3.7.8, 3.7.8, 3.7.9, 3.7.9, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.8.3, 3.8.3, 3.8.4, 3.8.4, 3.9.0, 3.9.0, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.0.2, 5.0.2, 5.0.3, 5.0.3, 5.0.4, 5.0.4
Skipped pre-versions: 3.0.0b1, 3.0.0b1, 3.0.0b2, 3.0.0b2, 3.0.2.dev0, 3.0.2.dev0, 3.0.2.dev1, 3.3.0.dev0, 3.8.0a1, 3.8.0a1, 3.8.0a2, 3.8.0a2
There are incompatible versions in the resolved dependencies:
flake8==5.0.4 (from -r requirements\development.in (line 12))
flake8>=3.7 (from flake8-simplify==0.19.3->-r requirements\development.in (line 20))
flake8<5.0.0 (from flake8-bugbear==22.8.23->-r requirements\development.in (line 14))
flake8!=3.2.0,>=3.0 (from flake8-comprehensions==3.10.0->-r requirements\development.in (line 15))
flake8<6,>=3.5 (from flake8-eradicate==1.3.0->-r requirements\development.in (line 17))The tools store packages in different areas on the file system. Pre-commit uses a cache area in the user folder, whereas pip-tools uses the usual site-packages area.
It should be noted that not all pre-commit hooks are written in Python for Python, so it makes sense that it has its own cache file system independent of pip. Therefore, not all pre-commit repositories have an entry in PyPI. They can simply be defined by a GitHub repository URL.
Lastly, browsing the internet looking for a solution I came across the following issue raised under the pip-tools project (dated 24 Jun 2021). The issue is still open and describes the exact same problem.
Just add to the pre-commit configuration file (.pre-commit-config.yaml). I have configured it to run at every commit at the pre-commit stage of git.
- repo: https://github.com/Stephen-RA-King/piptools-sync
rev: 0.3.1
hooks:
- id: piptools_syncInstalling by pip enables the package to be run from the command line at any time.
pip install piptools-syncPassing example.
piptools_sync............................................................PassedFailing example.
piptools_sync............................................................Failed
- hook id: piptools_sync
- exit code: 1
- files were modified by this hook
INFO:init:flake8 - piptools: 4.0.1 != pre-commit: 5.0.4Note: In this case piptools-sync will automatically update the pre-commit config file with pip-tools version. However, the commit with fail and will need to be re-run.
Passing example.
$ piptools_sync
Success! - pre-commit is in sync with piptoolsFailing example.
$ piptools_sync
flake8 - piptools: 4.0.1 != pre-commit: 5.0.4Note: In this case piptools-sync will automatically update the pre-commit config file with pip-tools version
For more examples and usage, please refer to the Wiki.
- Move some global variables into a separate settings file (toml).
- Settings configurable from the command line (and therefore configurable from the pre-commit.yaml file).
- Improve web request performance with asyncio / aiohttp libraries.
Stephen R A King : sking.github@gmail.com
Distributed under the MIT license. See
Created with Cookiecutter template: 
Digital Object Identifier:
