Should pytest fixtures stop being dependency injected?

[ssbarnea]

As we know, part of pytest design was to magically discover/load fixtures and make them available to tests, so test writer would not have to remember to import them.

That magic used was dependency injection and it was made very long time ago, far before python typing was introduced. Sadly,it does not play very well with typing because runtime checkers like mypy cannot really infer the types for test method signatures. Lack of it causes confusiones and usually breaks auto-complete in IDEs.

Inside #5981 it was suggested that maybe we should reconsider this architecture decision, so we do not cripple the typing support.

Please note that this poll result does not mean it would be acten on, think about it as a survey.

[RonnyPfannschmidt]

I use explicit type annotation just fine, jedi and a number of ides also have basic support for inferring

[The-Compiler]

Getting rid of the existing fixture mechanism is a non-starter, IMHO - it would be a major rewrite for anything using pytest. That massive code churn is hard to justify, and not going to be liked. Also, not everyone likes using types (I do!).

One thing I had in the back of my head for a while is class fixtures. In a nutshell:

• Allow @pytest.fixture on a class (with an __init__ taking other fixtures, or no arguments)
• Such fixtures are not auto-discovered like normal fixtures
• Instead, it gets injected whenever there's a type annotation (with associated import) for that class
• tbd: How to do teardown and such

This would be backwards compatible, but allow using imports and type annotations for fixtures where desired.

Probably should see how hard it is to implement and/or cook up a more detailed proposal at some point...

[RonnyPfannschmidt]

I'd much rather see nicely typed context managers than a new protocol

[nicoddemus]

I'd much rather see nicely typed context managers than a new protocol

Something like:

@contextmanager
def session_fixture(url: str):
   result = Session.connect()
   yield result
   result.disconnect()

def pytest_fixtures():
   return {"session": session_fixture}      

?

I see, but then it is basically what we have already with @pytest.fixture, except it is not auto discovered (the pytest_fixtures provides the fixtures explicitly).

But yeah, redesigning the fixture system to make it more flexible so plugins can customize it is a different and more complex topic.

[The-Compiler]

@nicoddemus Having to define a name defeats the whole purpose, in my eyes: Not having to refer to fixtures by some kind of "magical" global name, but importing a type from somewhere (nicely namespaced) and using the fixture by having an argument which is annotated by that type.

As for teardown, that would indeed be some kind of custom protocol. We could have some kind of pytest_teardown method called, for example. Or we could use __enter__ and __exit__ perhaps (i.e. making class fixtures context managers), which might or might not be what @RonnyPfannschmidt meant.

As for encouraging users to use @pytest.fixture in production code: Fair, but people could already do all kinds of pytest specific things in their production code (e.g. a pytest plugin). I don't think it's our job to prevent that.

[nicoddemus]

@nicoddemus Having to define a name defeats the whole purpose, in my eyes: Not having to refer to fixtures by some kind of "magical" global name, but importing a type from somewhere (nicely namespaced) and using the fixture by having an argument which is annotated by that type.

Fair. 👍

[RonnyPfannschmidt]

I think we need a Mindmap and a Design document, I'm under the impression our understanding and direction for those ideas display a level of diversity that dooms it unless we converge and experiment

We also should take a look at libraries like di and consider ways in which to bundle correlated fixtures in a consistent manner (failsafe for details of pytest selenium)

[nicoddemus]

Sadly,it does not play very well with typing because runtime checkers like mypy cannot really infer the types for test method signatures.

Not sure I follow this statement... mypy does not infer types for signatures AFAIK, if you don't add a type hint it is considered Any, not inferred automatically.

Lack of it causes confusiones and usually breaks auto-complete in IDEs.

IDEs (well at least PyCharm) infer the type of the fixtures in tests (because they have specific support for pytest). I believe VSCode does too.

Could you elaborate on both points please? I think it is important to clarify them because someone might "fly by" over the pool and leave with the impression that they can't use type annotations with pytest, or they are incompatible somehow (which is not true).

[ssbarnea]

Lets pick:

def test_foo(tmp_dir):
    tmp_dir.xxx = "x"

The IDE will not be able to tell you that .xxx does not exist on tmp_dir, because it assumes Any. As tmp_dir does not have a type, user needs to add it. Only when he adds the full signature, he will be able to get auto-complete to work and also the useful hint that .xxx is not really there.

Basically until you add types, you cannot really get useful help from tooling.

As most of our documentation examples do not have types, people copy/paste those and wonder what is that object. Is that string, is that a Path? (no need to answer me that, just trying to explain it).

Sadly I do not know what we can do to improve the situation other than advertising code using types almost everywhere. I am more than curious to see if someone comes with better ideas.

[nicoddemus]

Basically until you add types, you cannot really get useful help from tooling.

Oh I agree, but that's the same as with any other library: you will get the same problem if you don't add types to a function which receives a numpy array or a pandas data-frame. This is not pytest-specific, like the wording in the original post suggests.

The solution here is to just add the type to the fixture parameter:

def test_foo(tmp_dir: Path):
    tmp_dir.xxx = "x"

But I'm sure you know that.

Seems like you want (and sorry if I'm wrong) to change pytest to force users to use types, which I think is a non-starter, for the reasons @The-Compiler outlined.

What we can improve, I think, is having a dedicated page in the docs like "How to use types with pytest" or something, showing examples and explaining the current situation.

[nicoddemus]

Just to be clear: I love types and add types everywhere I can (including to tests in my recent projects), but I certainly don't think pytest should force everyone to use types.

[JacobCallahan]

This is a hard pass from me. One of the big advantages pytest offers over other test platforms is the "magic" of fixture dependency injection. Changing that behavior, to require explicit imports in order to benefit the subsection of users that also use types, would likely keep most users from upgrading to that pytest version.

I think a better way would be to recommend specific rules for fixture organization in a project. IDEs and tools like mypy could then make a best effort to discover fixtures based on these rules. This is the approach I'm taking with a project I'm working on that also needs to solve the fixture resolution problem.

[tgross35]

Out of curiosity - what would the imported version look like? The current "magic" way of course can't be removed: that being said, I'm in the camp that magic isn't real and providing an alternative certainly wouldn't be bad.

I haven't seen a concrete list of reasons (goals) that an import-related syntax might be preferred but here are a couple things I can think of:

• IDE support out of the box, rather than using plugins
• Static checkers can understand what's going on better (there is definitely value in running mypy on tests)
• (arguably) better code readability
• Potential namespace / scope benefits (e.g. easier fixture definition in different files, easy to find which fixture is used)

However, I'm having a bit of a tough time imagining how this might look practically. Importing the fixtures directly to use them has the slight disadvantage that property-like objects now need to be called as functions (I think, unless pytest can change the function definitions to objects. But that sounds messy for typing). Another downside is confusing during migration - global imported callable fixtures would have the same name as kwargs.

One random brainstorming thought was that @pytest.fixture could register fixtures as properties to a global importable class, maybe something like below. I actually like the appearance a lot and keeping fixtures in a fx namespace would help avoid backward-compatibility confusion (like above), as well as avoiding circular import issues or our-of-order definitions. This might not easily resolve the goals of IDE support and correct typing though, not sure the state of support for dynamically added objects.

# Current syntax
# my_fixture in conftest.py, my_local_fixture in this file

def test_something(my_fixture: str, my_local_fixture: int):
    assert my_fixture == "abcd"
    assert my_local_fixture == 4

# Potential new syntax
# Same fixture definitions as above assumed
# Plus nonlocal_fixture defined in tests/otherfile.py

from pytest import fx

# decorator optional, something to indicate can't use fixtures as arguments
@pytest.mark.strict_fixtures
def test_something():
    fix_str = fx.my_fixture. # property-like syntax possible
    fix_int = fx.my_local_fixture
    fix_other = fx.otherfile.nonlocal_fixture # Nonlocal fixtures accessible
    assert fix_str == "abcd"
    ...

Interested to see what the original idea looked like

[RonnyPfannschmidt]

@tgross35 any api for explicit fixtures would involve a way to sensibly do all fixture setup at call time, your example completely sabotages that

[tgross35]

@RonnyPfannschmidt Are you referring to having the fixture set up before entering the test_ function?

In any case, I'm mostly curious to see how the original direct import idea looked since I can't figure out how that would work.

[The-Compiler]

The original proposal in #5981 didn't propose imports for fixtures at all. I have a proposal in #9946 (comment) which would use classes for fixtures and type annotations, which I believe would work nicely with imports rather than a global(-ish) fixture namespace.

[ytimenkov]

First, I think there is nothing wrong with injecting fixtures as a parameters and otherwise it would be difficult to manage scopes (i.e. how to express session-scoped fixture?) with ordinary imports. The most important is to have type annotation for function parameter - how fixture can be used. This is pretty much covered in #9947.

It also looks like the discussion is between using type or parameter name as a key for injection. Right now pytest uses parameter name and type is ignored completely.

However, both type and parameter name are part of function interface (if we think about a test as an ordinary API function) - you can't change either freely without breaking users. And it doesn't really matter where pytest takes the actual values to be passed as parameters to the test.

There are other alternatives, each has pros and cons.

1. Use only type. For example, opyoid. Only declaring type and then specifying what provides instances of this type allows to name parameters freely and have, for example, 2 instances of the same type/class - e.g. if I need 2 temporary directories or DB connections.
2. Use both to some extent. This is how typer uses to inject command-line options. This is somewhat close to the proposed pytest + annotations, but leaves the name to be an external contract.

[burnpanck]

While this post seems to have some age, as it is still open, I would like to point out that the magic dependency injection also has significant downsides. As with any magic operating on a global namespace, it fails to respect modularity principles, which makes composition hard. The fact that typing tools have a problem with that is just one manifestation. The current design makes that namespace as global as it just could be, which has led to hard-to-debug bugs in the past (see e.g. #3966) and has led to the request of alternate ways of pulling fixtures (see e.g. #3834). Thus, anything that follows the rules of python should be preferred over domain-specific magic.

[nicoddemus]

I think while the discussion here is interesting, we can close this as pytest will not implement such a breaking change.

We are trying to improve the internal so plugins might be able to change and add to the existing fixture mechanism, so alternatives can be implemented outside the core.

Thanks everyone!

[RonnyPfannschmidt]

theres many ways to play this out - ultimately i'd love to see a way where fixture choice is optionally strictly controlled by annotations

when the fixture system was first invented, type annotations where not even in the planning
now a decade later type annotations are a very different story and building block

i have a rough idea for the necessary baseline refactoring to enable such usage (most of them will not add the capability)
i hope we ca line those out in the sprint

[burnpanck]

ultimately i'd love to see a way where fixture choice is optionally strictly controlled by annotations

That is exactly what I'm looking for. I don't care if the annotation is a decorator or a type annotation, but I would like to be able to follow variables, declarations and imports from the actual point where it is attached to some test code all the way to the implementation of the fixture, without having to consider a global scope of potential dependency injectors. After all, PEP-20 says "Explicit is better than implicit.".

[JacobCallahan]

Regarding plugins changing fixture behavior, plugin authors can strongly recommend a supported usage pattern for the plugin without the ability to turn off native pytest behavior. The native behavior can just exist and not be used if the plugin's users determine the plugin satisfies all their needs. There isn't a need to take a paternalistic position against your own users.

---

Regarding "Explicit is better than implicit", keep in mind the lines before and after:

• "Beautiful is better than ugly."
• "Simple is better than complex."

In my (very subjective) opinion, the current fixture pattern is both beautiful and simple (in most cases).

[RonnyPfannschmidt]

Indeed, it's just that in the last decade a different way to do it emerged and we need to start exploring and navigating it

[burnpanck]

The native behavior can just exist and not be used if the plugin's users determine the plugin satisfies all their needs. There isn't a need to take a paternalistic position against your own users.

This seems to miss the point. It's not the plugin author who wants to "forbid" their uses to use any other behaviour, but the plugin user who intends to select a specific fixture (using that plugin), without having control over the full codebase. The (current) native pytest behaviour means that changes to completely unrelated code may suddenly start to inject fixtures, without either author intending it. See #3966. Say you develop an application using two libraries "A" and "B". You test your application using testing helpers from both libraries (say, "A" is a networking library and you use their mock fixtures for the server, "B" is a GUI library and you use mock fixtures representing the GUI-user). "A" uses the new shiny plugin in their fixture, in the hope to achieve modularisation. "B" uses name-matching magic. At some point in time, both now have an internal fixture namedtest_client. Neither of them fails any tests, but you, as the application developer, who has no knowledge of the internal details of either library, now all of a sudden have failing tests, because the fixtures of "B" accidentally slip into "A", despite "A" using the new shiny plugin without name matching. Of, course, that depends on how that plugin interacts with pytest to achieve its goal, and I wanted to drive attention to that point.

I admit, name-matched fixtures are simple to write, and beautiful as long as you see through all involved code, globally. But they are extremely hard to read if the fixture definition is not in the same file, they don't provide any form of namespaceing, leading to ambiguity among which fixture is meant and break the rules of python name resolution. On the other hand, importing a fixture and applying it using either a decorator or an annotation to attach it to a test is also beautiful, and still simple enough IMHO.

[benjaoming]

If you look at the way that FastAPI uses dependency injection, you will also find a pattern that's compatible with typehints and type checkers.

Following the footsteps of FastAPI is certainly a proven track-record, so also seems safe and to repeat experience/knowledge that many developers might already have.

Previous:

@pytest.fixture
def my_fixture():
    return [1, 2, 3]

def test_something(my_fixture):
    assert sum(my_fixture) == 6

This could become something like:

from typing import Annotated, List
from pytest import Depends

def get_my_fixture() -> List[int]:
    return [1, 2, 3]


def test_something(my_fixture=Annotated(List[int], Depends(get_my_fixture))):
    assert sum(my_fixture) == 6

[benjaoming]

Yes, indeed

[burnpanck]

@RonnyPfannschmidt: The kind of decoupling you ask for can be easily achieved using the proposed pattern, and thats also what FastAPI, Litestar and quite a few other libraries are doing. The way that is done is the same as in other languages (e.g. next.js): You don't depend on an implementation object, but an API object that exposes just the "public part" of the API that shall be consumed by the dependant. Then, there is some kind of registration scheme to associate actual implementations with that API - in python, that registration could be fully automatic.

However, I would question the value of that "full decoupling" for the case of testing. What one usually wants to achieve with that sort of dependency injection is to decouple the usage from the implementation in order not to depend on implementation details! But in the case of test fixture, what is API and what is implementation? There are just two things that we could even consider: Side effects of the fixture, or the fixture return value provided to the test. In the case of the return value, the "API" of the fixture is usually not just the type of that return value, but really some properties of that value (i.e. I don't just want any HTTP connection client; I want one connected to the mock server!). Thus, that API object in the typical dependency injection scheme could be a type httpx_testing.ClientConnectedToMockServer rather than httpx.Client. Of course if the "API" is really the side-effects of the fixture, you can still communicate that through a specific type like httpx_testing.TestServerReservationHandle. So indeed, now we have associated the fixture "API" with a type, and we could use that type as annotation for the test.

This could in some cases indeed solve a few keystrokes. However, I question the value of that decoupling given that we now simply substituted a new name for the fixture. The only benefits of this indirection would come if there is a need to provide many different implementations of the fixture that have the same "API". Otherwise, you could just as well mention that dependency directly.

Note that from the point of view of decoupling "API" from implementation, the current name-based magic fails completely: It does not allow to swap in different implementations given the same API. It does not provide any clear specification of the API beyond a name. All it does, is that it avoids the use of an import statement by ditching namespaces and instead mandating one single global namespace for all test fixtures.

[burnpanck]

Actually, the proposed syntax is good for fixtures where we care for the "value", but if we only need a side-effect, decorators are the "standard" way to declare that info. So I vote for the following syntax towards the fixture user:

import typing
import pytest
from my_fixtures import fixture_for_side_effect, fixture_for_value

@pytest.uses_fixtures(fixture_for_side_effect)
def test_with_side_effect_fixture():
    pass

def test_with_value_fixture(fixture_value: typing.Annotated[list[int], pytest.FixtureDep(fixture_for_value)]):
    pass

Note that the example fixtures above (fixture_for_side_effect and fixture_for_value) could both be a direct link to fixture implementations (as it is today) or a reference to a fixture specification / API as discussed in the previous comment.

While the typing.Annotated[...] is a bit verbose indeed, it is standard and well-readable and is almost DRY; the only repetition ia in the value type. Half-joking proposition: someone should push the core python language for a feature like typing.DependencySpec(), with some protocol for type checkers to infer the value type from the dependency specification.

[RonnyPfannschmidt]

I consider this hijacking of scope and conflating of arguments hostile, so I will disengage this thread.

The proposed examples I posted still enable those shemes without exploding each single Test needing them.

The know issues of the fixture system where not subject of this discussion before as the intent is exploration of better ways (precisely because we have those issues).

[burnpanck]

@RonnyPfannschmidt I apologize. I did not intend to hijack the thread and certainly not to be hostile. Name matching wasn't part of the thread and shouldn't be.

The argument I wanted to make was specifically geared at figuring out what kind of "dependeny injection" we actually want - the type we already have, which does not separate API from implementation, or one that does.

Independent of which we choose, both may be architected such to address what I perceive is considered the main issue here: The lack of support from type checkers. So @benjaoming proposed a scheme using Depends which does solve the type-checker issue, while largely retaining the other behaviours of the existing system. It is more verbose to type though. You suggested to use a type alias (e.g. CarryDepends) to shorten the typing, and I think that is a fine proposition.

You also suggested a separate facility (MatchType) which would provide a form of "real dependency injection", where the use site does not link to the implementation of the dependency, but instead on an abstract specification of the functionality that's needed. I agree with @benjaoming that using just the type of the value as the specification is actually counter-productive. It hardly specifies well-enough what you actually expect from that fixture providing the type. However, I believe this can be fixed:

from typing import Annotated
from pytest import Depends, fixture, abstract_fixture

@abstract_fixture
def my_abstract_fixture() -> list[int]
     pass

@fixture(name=None, implements=abstract_fixture)
def my_fixture() -> list[int]:
   return [1,2,3]

def test_something(my_fixture: Annotated[list[int],Depends(my_abstract_fixture)]):
     assert sum(my_fixture) == 6

# or, of course, using a type alias
type WithAbstractFixture =Annotated[list[int],Depends(my_abstract_fixture)]

def test_something_else(my_fixture: WithAbstractFixture):
     assert sum(my_fixture) == 6