New parametrized type annotations for run_in_threadpool incompatible with generic functions

[florimondmanca]

After #1383 was released, run_in_threadpool() raises a mypy error for generic functions.

Reproduction code:

from typing import TypeVar

from starlette.concurrency import run_in_threadpool

T = TypeVar("T")


async def func(value: T) -> T:
    return value


async def main() -> None:
    val = await run_in_threadpool(func, 1)
    reveal_type(val)

$ mypy example.py
example.py:13: error: Argument 2 to "run_in_threadpool" has incompatible type "int"; expected "T"  [arg-type]
example.py:14: error: Value of type "Coroutine[Any, Any, T]" must be used  [unused-coroutine]
example.py:14: note: Are you missing an await?
example.py:14: note: Revealed type is "typing.Coroutine[Any, Any, T`-1]"
Found 2 errors in 1 file (checked 1 source file)

The concrete situation where I encountered this was wrapping a glob.glob() call: glob() is typed as accepting AnyStr, so mypy complains when I do run_in_threadpool(glob.glob, "some/pattern/*.md") with the same kind of error: "Expected AnyStr".

I might recommend reverting the part of #1383 related to run_in_threadpool. It would be a neat addition if ParamSpec was doing the right thing here, but it isn't. 😕 I reckon this might very well be a mypy issue, though, we'd need to check the ParamSpec PEP for generics first.

[Kludex]

Just for reference, this is your original issue:

import glob
from typing import TypeVar, ParamSpec, Callable


T = typing.TypeVar("T")
P = ParamSpec("P")

async def run_in_threadpool(
    func: typing.Callable[P, T], *args: P.args, **kwargs: P.kwargs
) -> T:
    if kwargs:  # pragma: no cover
        # run_sync doesn't accept 'kwargs', so bind them in here
        func = functools.partial(func, **kwargs)
    return await anyio.to_thread.run_sync(func, *args)

async def main() -> None:
    await run_in_threadpool(glob.glob, "*")

This leads to:

starlette/concurrency.py:48: error: Argument 2 to "run_in_threadpool" has incompatible type "str"; expected "AnyStr"  [arg-type]

I've asked about this on the python/typing Gitter channel. Let's see if there's any reply. 🙏

We can already turn this into an issue if you want. 😗

[Kludex]

This is already an issue on mypy: python/mypy#12278 (comment)

As they are positive on implementing this, we could either not revert and wait for them to fix it, or revert the type annotation on run_in_threadpool and enhance it later on. I'm fine with either. 👍

[Kludex]

After some years, there's activity: python/mypy#12278 🙏

[Kludex]

This was solved. 👍