A RuntimeError: Stream consumed error is raised when request.body() is read in the custom middleware

[theredfoxlee]

Hi,

I encountered the following issue while using Starlette today :)

Issue description

A RuntimeError: Stream consumed error is raised when request.body() is read in the custom middleware (1) after call_next, given that a route uses request.body().

How to reproduce the issue?

Code

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware import Middleware

async def testpage(request):
    await request.body()
    return JSONResponse({'hello': 'world'})


class CustomMiddleware(BaseHTTPMiddleware):

    async def dispatch(self, request, call_next):
        response = await call_next(request)
        await request.body()  # THIS WILL RAISE -> RuntimeError: Stream consumed
        return response


app = Starlette(
    debug=True,
    routes=[Route('/', testpage, methods=["PUT"])],
    middleware=[Middleware(CustomMiddleware)]
)

Dependencies

starlette==0.37.2
uvicorn==0.29.0

Commands

• 1st terminal: uvicorn example:app
• 2nd terminal: curl -XPUT http://127.0.0.1:8000 -d '123'

Known workarounds

1. Use request.body() at least once before call_next to ensure that the _body is cached.

I prefer not to use a workaround, as I only want to read the body in the middleware conditionally, based on the response, rather than always.

---

This is my first time using starlette. Is this usage incorrect, or is it a bug?

[theredfoxlee]

I believe it's a bug. May I move it to the issues?

[theredfoxlee]

The issue here is that we're using starlette.requests.Request inside the route, which seems not to propagate the read .body to starlette.middleware.base._CachedRequest used in the middleware.

[ex3cv]

Hi @theredfoxlee

Just move await request.body() to be called first in your method. That should do the trick. ;-)

async def dispatch(self, request, call_next):
    await request.body()
    response = await call_next(request)
    return response

[theredfoxlee]

Hi @ex3cv, thanks, but I've already mentioned that in Known workarounds. It doesn't solve the bug; it avoids it :/

[adriangb]

This isn't a bug, this is just how things work with ASGI / streaming data.

If you don't want to first read (and thus cache) the request body in the middleware I recommend you do something like this:

from random import randint

from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
from starlette.requests import Request
from starlette.responses import Response
from starlette.routing import Route


async def read_body_cached(request: Request) -> bytes:
    request.scope['cached_body'] = body = await request.body()
    return body


async def endpoint(request: Request) -> Response:
    if randint(1, 1) > 0.5:  # some condition you want
        body = await read_body_cached(request)
    else:
        body = await request.body()
    return Response(body)


async def middleware(request: Request, call_next: RequestResponseEndpoint) -> Response:
    response = await call_next(request)
    body: bytes | None = request.scope.get('cached_body')
    if body:
        response.headers['X-Cached-Body'] = body.decode()
    return response


app = Starlette(
    debug=True, routes=[Route('/', endpoint, methods=['PUT'])], middleware=[Middleware(BaseHTTPMiddleware, dispatch=middleware)]
)

if __name__ == '__main__':
    from starlette.testclient import TestClient

    client = TestClient(app)

    response = client.put('/', content=b'Hello, world!')
    assert response.status_code == 200
    assert response.text == 'Hello, world!'
    assert response.headers['X-Cached-Body'] == 'Hello, world!'

This assumes that you know that there is no middleware that mutates the body (e.g. decompression), etc.