Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use uv to install dependencies #10507

Merged
merged 4 commits into from
Aug 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,795 changes: 2,795 additions & 0 deletions .github/.test_durations

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .github/workflows/benchmarks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
steps:
- uses: actions/setup-python@v5
with:
python-version: "3.11"
python-version: "3.12"
- uses: iterative/dvc-bench@main
with:
pytest_options: "-k 'test_init or test_help'"
5 changes: 3 additions & 2 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,15 @@ jobs:
with:
python-version: 3.12

- run: python -m pip install --upgrade pip setuptools_scm build twine
- uses: hynek/setup-cached-uv@v2
- run: uv pip install --upgrade setuptools_scm build twine --system

- name: Force version for Test PyPI upload
if: ${{ !startsWith(github.ref, 'refs/tags') }}
run: echo version=$(python -m setuptools_scm | awk -F+ '{print $1}' | tail -1) >> $GITHUB_ENV

- run: echo 'PKG = "pip"'>dvc/_build.py
- run: python -m build
- run: python -m build --installer=uv
env:
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_DVC: ${{ env.version }}
- run: twine check --strict dist/*
Expand Down
15 changes: 8 additions & 7 deletions .github/workflows/plugin_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04]
pyv: ["3.10"]
os: [ubuntu-latest]
pyv: ["3.12"]
plugin: ["dvc-s3"]

steps:
Expand All @@ -37,16 +37,17 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyv }}
cache: pip

- uses: hynek/setup-cached-uv@v2
with:
cache-dependency-path: |
dvc/pyproject.toml
${{ matrix.plugin }}/setup.cfg
${{ matrix.plugin }}/pyproject.toml

- name: Install plugin + DVC@PR
run: |
pip install --upgrade pip wheel
pip install "./dvc[testing]"
pip install -e "./${{ matrix.plugin }}[tests]"
uv pip install "./dvc[testing]" --system
uv pip install -e "./${{ matrix.plugin }}[tests]" --system

- name: Run plugin tests
timeout-minutes: 15
Expand Down
75 changes: 64 additions & 11 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,68 @@ permissions:
contents: read

jobs:
lint:
timeout-minutes: 10
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: hynek/setup-cached-uv@v2
with:
cache-dependency-path: pyproject.toml
- run: uv pip install -e ".[dev]" --system
env:
UV_LINK_MODE: ${{ runner.os == 'Windows' && 'symlink' || 'hardlink' }}

- name: Cache mypy
uses: actions/cache@v4
with:
path: .mypy_cache
key: mypy-${{ runner.os }}-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}
- uses: pre-commit/action@v3.0.1

tests:
timeout-minutes: 50
timeout-minutes: 20
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, windows-latest, macos-latest]
os: [ubuntu-latest, macos-latest]
pyv: ["3.9", "3.10", "3.11", "3.12"]

include:
- os: windows-latest
pyv: "3.9"
pytestargs: "--splits 4 --group 1"
- os: windows-latest
pyv: "3.9"
pytestargs: "--splits 4 --group 2"
- os: windows-latest
pyv: "3.9"
pytestargs: "--splits 4 --group 3"
- os: windows-latest
pyv: "3.9"
pytestargs: "--splits 4 --group 4"
- os: windows-latest
pyv: "3.12"
pytestargs: "--splits 4 --group 1"
- os: windows-latest
pyv: "3.12"
pytestargs: "--splits 4 --group 2"
- os: windows-latest
pyv: "3.12"
pytestargs: "--splits 4 --group 3"
- os: windows-latest
pyv: "3.12"
pytestargs: "--splits 4 --group 4"
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -33,20 +86,20 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyv }}
cache: pip
- uses: hynek/setup-cached-uv@v2
with:
cache-dependency-path: pyproject.toml
- name: install
run: |
pip install --upgrade pip wheel
pip install -e ".[dev]"
- uses: pre-commit/action@v3.0.1
- run: uv pip install -e ".[dev]" pytest-split --system
env:
UV_LINK_MODE: ${{ runner.os == 'Windows' && 'symlink' || 'hardlink' }}

- name: run tests
timeout-minutes: 40
env:
PYTHONUTF8: 1
run: >
pytest -n=logical --timeout=300 --durations=100
--cov --cov-report=xml --cov-report=term
pytest ${{ matrix.pytestargs }} -n=logical --timeout=300 --durations=0
--cov --cov-report=xml --cov-report=term --durations-path=./.github/.test_durations
- name: upload coverage report
uses: codecov/codecov-action@v3
with:
Expand Down
23 changes: 17 additions & 6 deletions tests/func/test_daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from collections import defaultdict
from collections.abc import Iterator
from contextlib import contextmanager, suppress
from http.server import HTTPServer, SimpleHTTPRequestHandler
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
from threading import Thread
from typing import ClassVar
Expand Down Expand Up @@ -41,7 +41,7 @@


def make_request_handler():
class RequestHandler(SimpleHTTPRequestHandler):
class RequestHandler(BaseHTTPRequestHandler):
# save requests count for each method
hits: ClassVar[dict[str, int]] = defaultdict(int)

Expand Down Expand Up @@ -80,7 +80,10 @@ def server():
thread = Thread(target=httpd.serve_forever)
thread.daemon = True
thread.start()
yield httpd
try:
yield httpd
finally:
httpd.shutdown()


def test_analytics(tmp_path, server):
Expand Down Expand Up @@ -110,9 +113,13 @@ def test_analytics(tmp_path, server):
pid = int(match.group(1).strip())

with suppress(psutil.NoSuchProcess):
psutil.Process(pid).wait(timeout=10)
psutil.Process(pid).wait(timeout=30)
skshetry marked this conversation as resolved.
Show resolved Hide resolved

log_contents = logfile.read_text(encoding="utf8")
expected_line = (f"Process {pid} " if os.name != "nt" else "") + "exiting with 0"
assert expected_line in log_contents

assert not os.path.exists(report_file)
assert f"Process {pid} exiting with 0" in logfile.read_text(encoding="utf8")
assert server.RequestHandlerClass.hits == {"POST": 1}


Expand Down Expand Up @@ -142,7 +149,11 @@ def test_updater(tmp_dir, dvc, server):

with suppress(psutil.NoSuchProcess):
psutil.Process(pid).wait(timeout=10)
assert f"Process {pid} exiting with 0" in logfile.read_text(encoding="utf8")

log_contents = logfile.read_text(encoding="utf8")
expected_line = (f"Process {pid} " if os.name != "nt" else "") + "exiting with 0"
assert expected_line in log_contents

assert server.RequestHandlerClass.hits == {"GET": 1}
# check that the file is saved correctly
updater_file = Path(dvc.tmp_dir) / Updater.UPDATER_FILE
Expand Down
Loading