Skip to content

Commit

Permalink
Improve speed and stability of Python 3.12 tests in canary build
Browse files Browse the repository at this point in the history
Python 3.12 introduced a new (much faster) way of tracking and
monitoring execution of python code by tools like coverage tracking
using sysmon (PEP 669). This however also apparently heavily impacted
performance of coverage tracking for Python 3.12 when PEP 669 is not
used. The coverage library since 7.4.0 has an experimental support
for PEP 669 that can be enabled with COVERAGE_CORE=sysmon env variable
and a number of users confirmed it fixes the problem.

We are using 7.4.4 coverage already so we should enable this mode
to speed up our coverage tracking. That should also allow us to
remove databricks from excluded providers.

See databricks/databricks-sql-python#369
for databricks case and nedbat/coveragepy#1665
for coverage bug.
  • Loading branch information
potiuk committed Mar 15, 2024
1 parent 4c1ad2a commit 6a4488a
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions dev/breeze/src/airflow_breeze/commands/testing_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -841,6 +841,8 @@ def helm_tests(
python_version=shell_params.python,
helm_test_package=helm_test_package,
)
if sys.version_info >= (3, 12):
env["COVERAGE_CORE"] = "sysmon"
cmd = ["docker", "compose", "run", "--service-ports", "--rm", "airflow", *pytest_args, *extra_pytest_args]
result = run_command(cmd, check=False, env=env, output_outside_the_group=True)
fix_ownership_using_docker()
Expand Down

0 comments on commit 6a4488a

Please sign in to comment.