Loading files for tests

[mattgwwalker]

How do I load a data file for a test? Specifically, how do I load the file given that I do not know the current working directory?

I would like to test that my code correctly opens an audio file. Imagine the following layout:

rootdir
  - project
  - tests
    - test_example.py
  - examples
    - example_audio.ogg

"pytest" can be run from within rootdir or, often, from with the tests directory itself.

Currently we're resolving this issue with the following line inserted at the top of every test file:

os.chdir(os.path.dirname(__file__))

Then we know the current working directory, and we can load files relative to it:

open("../examples/example_audio.ogg")

Is there a better way to do this? Forgetting the chdir line leads to hard-to-track down errors as it seems the lines are run when the tests are found rather than just before they are executed. If we were to add a sub-directory of tests then I would suspect our current technique would fail.

I was expecting to find a general solution to be something like:

open(pytest.rootdir / "examples/example_audio.ogg")

But I can't find any relevant documentation or discussions. Does anyone have any suggestions for how to resolve this or recommendations for articles I should read?

Thanks,

Matthew

[graingert]

I tend to use importlib.resources

rootdir
  - src
    - project
  - tests
    - __init__.py
    - test_example.py
    - resources
      - __init__.py
      - audio.ogg

from . import resources

def test_audio():
    with importlib.resources.path(resources, "audio.ogg") as path:
        assert sut(path) = "expected"

[mattgwwalker]

Thanks for your super quick reply.

It's a very interesting approach. As I understand it, it's for Python 3.7+ only. I suppose before then the approach was based on pkg_resources?

We're currently sharing the audio data files between the tests and examples. The examples just make the assumption that the user is running them from the examples directory and thus all file loading is from the current working directory.

To use your importlib.resources suggestion we could copy all the audio files to the tests directory. However, there are a few of those files and duplication's never the first-choice option. Given this additional information, do you have any further suggestions?

Also, for my interest, what's the sut() function in your assert? I had no luck Googling it.

Thanks again,

Matthew

[graingert]

You can use the https://pypi.org/project/importlib-resources/ backport

Also, for my interest, what's the sut() function in your assert? I had no luck Googling it.

sut stands for System Under Test, it's a metasyntactic variable name

[nicoddemus]

I recommend pytest-datadir, I use it all the time. 👍

[mattgwwalker]

Thanks @nicoddemus, I have just seen your suggestion.

I was going to report that I had success with adding the following into a conftest.py that I added into the tests directory:

import pytest

def pytest_configure(config):
    pytest.rootdir = config.rootdir 

I was then able to specify a filename in a test using:

filename = str(pytest.rootdir / "examples/example_audio.ogg")

This approach seems to work well for me. Do you see any negatives?

Thanks again,

Matthew

[mattgwwalker]

One negative of the solution I proposed is that mypy doesn't like it much. It complains with:

error: Module has no attribute "rootdir"

[nicoddemus]

This approach seems to work well for me. Do you see any negatives?

If you want a directory relative to the root dir, go for it! I would suggest setting that global variable somewhere else other than the pytest module though, as that might confuse someone reading your code.

[The-Compiler]

Note you could also use the pytestconfig fixture from a test (or fixture) and use pytestconfig.rootdir instead.