Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Python 3.13 is officially supported #2518

Merged
merged 23 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8f8a5be
test: Start testing with Python 3.13
edgarrmondragon Jul 2, 2024
6c3ab3b
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Aug 28, 2024
3e34b18
Let tests run
edgarrmondragon Aug 28, 2024
97d04e5
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Sep 10, 2024
527961f
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Sep 26, 2024
932a293
Bump duckdb to get Python 3.13 wheels
edgarrmondragon Sep 26, 2024
d1bf151
Update xfail markers
edgarrmondragon Sep 26, 2024
8f516fb
Ignore SQLite `ResourceWarnings`
edgarrmondragon Sep 26, 2024
3a8df2e
Update templates
edgarrmondragon Sep 26, 2024
32a553a
Update more xfail markers
edgarrmondragon Sep 26, 2024
8541ff8
There are some XPASS
edgarrmondragon Sep 26, 2024
be2cb48
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Oct 2, 2024
1bcd297
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Oct 8, 2024
17de2c8
Update max Py version in `--about`
edgarrmondragon Oct 8, 2024
7f3cf75
Use latest Python where possible
edgarrmondragon Oct 8, 2024
ab85c05
Allow some sessions to run on any Python
edgarrmondragon Oct 8, 2024
c2c04e9
Fix external tests
edgarrmondragon Oct 8, 2024
e1c014a
Restore 3.12 as main Python version
edgarrmondragon Oct 8, 2024
63558aa
Let mypy and deps run
edgarrmondragon Oct 8, 2024
ec3c184
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Oct 9, 2024
eedde87
Merge branch 'main' into edgarrmondragon/feat/python3.13
edgarrmondragon Oct 17, 2024
a03bae8
Use default latest Python for extra sessions
edgarrmondragon Oct 17, 2024
c76b915
Revert "Use default latest Python for extra sessions"
edgarrmondragon Oct 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ body:
description: Version of Python you are using
options:
- "NA"
- "3.13"
- "3.12"
- "3.11"
- "3.10"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/api-changes.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.12
python-version: 3.x

- name: Install tools
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codspeed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.12
python-version: 3.x
architecture: x64

- name: Install poetry
Expand Down
17 changes: 5 additions & 12 deletions .github/workflows/cookiecutter-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,8 @@ env:

jobs:
lint:
name: Cookiecutter E2E Python ${{ matrix.python-version }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
include:
- { python-version: "3.12", os: "ubuntu-latest" }

name: Cookiecutter E2E Python
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Upgrade pip
Expand All @@ -50,8 +44,7 @@ jobs:

- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
architecture: x64
python-version: 3.x
cache: 'pip'
cache-dependency-path: 'poetry.lock'

Expand All @@ -69,12 +62,12 @@ jobs:

- name: Run Nox
run: |
nox --python=${{ matrix.python-version }} --session=test_cookiecutter
nox --session=test_cookiecutter

- uses: actions/upload-artifact@v4
if: always()
with:
name: cookiecutter-${{ matrix.os }}-py${{ matrix.python-version }}
name: cookiecutter-ubuntu-latest-py3x
path: |
/tmp/tap-*
/tmp/target-*
Expand Down
15 changes: 11 additions & 4 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,17 @@ jobs:
matrix:
session: [tests]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
sqlalchemy: ["2"]
include:
- { session: tests, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "1" }
- { session: doctest, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }
- { session: tests, python-version: "3.13", os: "ubuntu-latest", sqlalchemy: "1" }
- { session: doctest, python-version: "3.13", os: "ubuntu-latest", sqlalchemy: "2" }
- { session: mypy, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }
- { session: deps, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }

Expand All @@ -64,6 +70,7 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true

- name: Upgrade pip
env:
Expand Down Expand Up @@ -152,7 +159,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
python-version: '3.x'

- name: Upgrade pip
env:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/version_bump.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,7 @@ jobs:

- uses: actions/setup-python@v5
with:
python-version: "3.12"
architecture: x64
python-version: "3.x"

- name: Bump version
id: cz-bump
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
license = "Apache-2.0"
{%- if cookiecutter.variant != "None (Skip)" %}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This file can be used to customize tox tests as well as other test frameworks like flake8 and mypy

[tox]
envlist = py3{9,10,11,12}
envlist = py3{9,10,11,12,13}
requires =
tox>=4.19

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
license = "Apache-2.0"
{%- if cookiecutter.variant != "None (Skip)" %}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This file can be used to customize tox tests as well as other test frameworks like flake8 and mypy

[tox]
envlist = py3{9,10,11,12}
envlist = py3{9,10,11,12,13}
requires =
tox>=4.19

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
license = "Apache-2.0"
{%- if cookiecutter.variant != "None (Skip)" %}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This file can be used to customize tox tests as well as other test frameworks like flake8 and mypy

[tox]
envlist = py3{9,10,11,12}
envlist = py3{9,10,11,12,13}
requires =
tox>=4.19

Expand Down
27 changes: 23 additions & 4 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import shutil
import sys
import tempfile
import typing as t
from pathlib import Path

import nox
Expand All @@ -23,7 +24,14 @@
COOKIECUTTER_REPLAY_FILES = list(Path("./e2e-tests/cookiecutters").glob("*.json"))

package = "singer_sdk"
python_versions = ["3.12", "3.11", "3.10", "3.9", "3.8"]
python_versions = [
"3.13",
"3.12",
"3.11",
"3.10",
"3.9",
"3.8",
]
main_python_version = "3.12"
locations = "singer_sdk", "tests", "noxfile.py", "docs/conf.py"
nox.options.sessions = (
Expand All @@ -35,7 +43,7 @@
)

poetry_config = nox.project.load_toml("pyproject.toml")["tool"]["poetry"]
test_dependencies = poetry_config["group"]["dev"]["dependencies"].keys()
test_dependencies: dict[str, t.Any] = poetry_config["group"]["dev"]["dependencies"]
typing_dependencies = poetry_config["group"]["typing"]["dependencies"].keys()


Expand All @@ -53,7 +61,18 @@ def mypy(session: nox.Session) -> None:
@nox.session(python=python_versions)
def tests(session: nox.Session) -> None:
"""Execute pytest tests and compute coverage."""
session.install(".[faker,jwt,parquet,s3]")
extras = [
"faker",
"jwt",
"parquet",
"s3",
]

if session.python == "3.13":
# https://github.com/apache/arrow/issues/43519
extras.remove("parquet")

session.install(f".[{','.join(extras)}]")
session.install(*test_dependencies)

sqlalchemy_version = os.environ.get("SQLALCHEMY_VERSION")
Expand Down Expand Up @@ -95,7 +114,7 @@ def benches(session: nox.Session) -> None:
)


@nox.session(name="deps", python=python_versions)
@nox.session(name="deps", python=main_python_version)
def dependencies(session: nox.Session) -> None:
"""Check issues with dependencies."""
session.install(".[s3,testing]")
Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,8 @@ filterwarnings = [
"ignore:No records were available to test:UserWarning",
# https://github.com/meltano/sdk/issues/1354
"ignore:The function singer_sdk.testing.get_standard_tap_tests is deprecated:DeprecationWarning",
# TODO: Address this SQLite warning in Python 3.13+
"ignore::ResourceWarning",
]
log_cli_level = "INFO"
markers = [
Expand Down
1 change: 1 addition & 0 deletions samples/sample_tap_dummy_json/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
license = "Apache-2.0"

Expand Down
2 changes: 1 addition & 1 deletion samples/sample_tap_dummy_json/ruff.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
src = ["tap_dummyjson"]
target-version = "py38"
target-version = "py39"

[lint]
ignore = [
Expand Down
4 changes: 2 additions & 2 deletions samples/sample_tap_dummy_json/tox.ini
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This file can be used to customize tox tests as well as other test frameworks like flake8 and mypy

[tox]
envlist = py{38,39,310,311,312}
envlist = py3{9,10,11,12,13}
isolated_build = true

[testenv]
Expand All @@ -13,7 +13,7 @@ commands =
[testenv:pytest]
# Run the python tests.
# To execute, run `tox -e pytest`
envlist = py{38,39,310,311,312}
envlist = py3{8,9,10,11,12,313}
commands =
poetry install -v
poetry run pytest
9 changes: 6 additions & 3 deletions samples/sample_target_parquet/parquet_target_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@

import typing as t

import pyarrow as pa
import pyarrow.parquet as pq

from singer_sdk.sinks import BatchSink

try:
import pyarrow as pa
import pyarrow.parquet as pq
except ImportError:
pass


def json_schema_to_arrow(schema: dict[str, t.Any]) -> pa.Schema:
"""Convert a JSON Schema to an Arrow schema.
Expand Down
2 changes: 1 addition & 1 deletion singer_sdk/about.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

# Keep these in sync with the supported Python versions in pyproject.toml
_PY_MIN_VERSION = 8
_PY_MAX_VERSION = 12
_PY_MAX_VERSION = 13


def _get_min_version(specifiers: SpecifierSet) -> int:
Expand Down
13 changes: 13 additions & 0 deletions tests/contrib/test_batch_encoder_parquet.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,23 @@

from __future__ import annotations

import sys
import typing as t

import pytest

from singer_sdk.contrib.batch_encoder_parquet import ParquetBatcher
from singer_sdk.helpers._batch import BatchConfig, ParquetEncoding, StorageTarget

if t.TYPE_CHECKING:
from pathlib import Path


@pytest.mark.xfail(
sys.version_info >= (3, 13),
reason="Parquet not supported on Python 3.13 due to PyArrow incompatibility",
strict=True,
)
def test_batcher(tmp_path: Path) -> None:
root = tmp_path.joinpath("batches")
root.mkdir()
Expand All @@ -30,6 +38,11 @@ def test_batcher(tmp_path: Path) -> None:
assert batches[0][0].endswith(".parquet")


@pytest.mark.xfail(
sys.version_info >= (3, 13),
reason="Parquet not supported on Python 3.13 due to PyArrow incompatibility",
strict=True,
)
def test_batcher_gzip(tmp_path: Path) -> None:
root = tmp_path.joinpath("batches")
root.mkdir()
Expand Down
4 changes: 2 additions & 2 deletions tests/core/test_about.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,8 @@ def test_get_supported_pythons_sdk():
"specifiers,expected",
[
(">=3.7,<3.12", ["3.7", "3.8", "3.9", "3.10", "3.11"]),
(">=3.7", ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]),
(">3.7", ["3.8", "3.9", "3.10", "3.11", "3.12"]),
(">=3.7", ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]),
(">3.7", ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]),
(">3.7,<=3.11", ["3.8", "3.9", "3.10", "3.11"]),
],
)
Expand Down
6 changes: 6 additions & 0 deletions tests/samples/test_target_parquet.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from __future__ import annotations

import shutil
import sys
import uuid
from pathlib import Path

Expand All @@ -23,6 +24,11 @@
)


@pytest.mark.xfail(
sys.version_info >= (3, 13),
reason="Parquet not supported on Python 3.13 due to PyArrow incompatibility",
raises=NameError,
)
class TestSampleTargetParquet(StandardTests):
"""Standard Target Tests."""

Expand Down
6 changes: 6 additions & 0 deletions tests/samples/test_target_sqlite.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import json
import sqlite3
import sys
import typing as t
from copy import deepcopy
from io import StringIO
Expand Down Expand Up @@ -367,6 +368,11 @@ def test_sqlite_process_batch_message(
assert cursor.fetchone()[0] == 4


@pytest.mark.xfail(
sys.version_info >= (3, 13),
reason="Parquet not supported on Python 3.13 due to PyArrow incompatibility",
strict=True,
)
def test_sqlite_process_batch_parquet(
sqlite_target_test_config: dict,
sqlite_sample_target_batch: SQLiteTarget,
Expand Down
Loading