Sentry <> Starlette: Error Tracking and Performance Monitoring

[smeubank]

Announcement:

Sentry has delivered a dedicated integration for Starlette!

What is available today?

• The integration is available with manual detection today - and will be automatic in the near future
• Check our docs here: Getting Started
• Project onboarding experience:

What we would like to feedback on?

Any and all feedback, issues, PRs are welcome! We will respond in a timely manner on the sentry-python SDK repo

• What are your use cases, that are most important to you?
• Is there specific context for error tracking or performance monitoring that you would like for us to cover?
• Spans for all the middleware is available for Starlette. Can you confirm how this is working for you?

[erewok]

This looks promising: we use Sentry at my work and I am a fan of their services.

We have written our own Prometheus-middleware for tracking latencies and request success/failure rates, and I had a question about your comment that:

Spans for all the middleware is available for Starlette

Does this automatically include performance information (I have seen some mention of this in general from Sentry recently) or response times or other interesting data like that? Does Sentry also offer waterfall graphs (a la Jaeger or other opentelemetry tooling) and is that what you meant by spans here?

Thanks again!

[smeubank]

Hi @erewok

Spans are sent in an array, inside transaction events payloads, which power the performance product. You can check out our SDK developer docs on transaction payloads, if you want to go farther down the rabbit hole.

But simply, yes this includes performance information. I think in this case we are talking about distributed tracing, in which case we have traces, made up of transactions, which are made up of span(s)

The docs are meaty, but basically there are some graphs there which tell the story simply.

And yes, specifically sentry offers trace view with waterfall graphs within the Performance product.

If you want any more help to onboard to our Performance product please let me know!

[erewok]

That looks incredible! Thanks for the information.

[tomchristie]

Fantastic work. 🌟

One actionable item, and one item for consideration here..

• A pull request adding this to the third party docs would help highlight the package... https://www.starlette.io/third-party-packages/
• I also wonder if it'd make sense to include a documentation change on https://www.starlette.io/exceptions/ - we don't typically have third party packages listed in those sections, but sentry's tooling here provides such an essential part of production deployments, that I think it could be valid to highlight it prominently in this way.

[smeubank]

Both are great suggestion! Thank you @tomchristie sharing with the team now

[antonpirker]

• The PR for third party docs is a good idea. I will create one after my vacation (in three weeks)
• And mentioning the sentry integration in the exceptions page would be very cool! Could we add a link to our setup guide [1] there?

1: https://docs.sentry.io/platforms/python/guides/starlette/

[tomchristie]

Could we add a link to our setup guide there?

I'd suggest we determine together as a team against an individual pull request what we consider appropriate.

(It seems like a decent idea to me.)

[smeubank]

Anton and I are both on holidays at the moment. Just looping in @sl0thentr0py

Adding mentions and possible links to the pages in starlette sounds agreed. We should probably also ensure in our sentry docs if we are missing context &/or links to help starlette devs back to any specific locations in your docs. (Now I'm confusing myself on the back and forth of docs 😅)

[smeubank]

@antonpirker fyi, happy for your input on wording and links. For correctness and relevance

#2063

#2062

[Kludex]

Not sure if it's relevant, but on WebSocket's you'll have always two exceptions being raised:

import os

import sentry_sdk
from sentry_sdk.integrations.starlette import StarletteIntegration
from starlette.applications import Starlette
from starlette.routing import WebSocketRoute
from starlette.websockets import WebSocket

SENTRY_DSN = os.getenv("SENTRY_DSN")

sentry_sdk.init(
    dsn=SENTRY_DSN,
    integrations=[StarletteIntegration()],
    # Set traces_sample_rate to 1.0 to capture 100%
    # of transactions for performance monitoring.
    # We recommend adjusting this value in production,
    traces_sample_rate=1.0,
)

class CustomWebSocketException(Exception):
    ...


async def websocket_error(websocket: WebSocket):
    await websocket.accept()
    raise CustomWebSocketException()


app = Starlette(routes=[WebSocketRoute("/sentry-ws", websocket_error)])

[Kludex]

Again, not sure if it's relevant... But the WebSockets transaction is using the endpoint style, even when "url" is set.

[Kludex]

As a note... If there's an error handler (500 or Exception) on the application, the package still sends the error to Sentry. This is great!

This is not the case on Flask when handling Exception (it's when handling 500 tho)! 👀

[Kludex]

The "App Only" option doesn't work as expected, I guess? I see the full trace from Starlette. I would expect only the main.py being presented there. 🤔

[Kludex]

That's what I could find on the Error Tracking. The application used: https://github.com/Kludex/starlette-sentry. 🙏

My trial expires in three days, can I have a free plan to do further testing on performance? 👀

[antonpirker]

Hey @Kludex! Thanks for the amazing testing work!
There is the "Developer" account [1] that has some free quote (I guess 50k errors a month) have you already depleted this?

1: https://sentry.io/pricing/

[Kludex]

I don't know how to switch to that... "Your free trial has ended. You may contact support to request another trial."

[antonpirker]

Can you give me your username or email you use for sentry? (sent to anton dot pirker at sentry io ) Then I can ask around to get you more quota

[antonpirker]

We had a bug report [1] about an app that has a Starlette middleware and Sentry enabled and when trying to read the request body in the middleware it hangs because Sentry already read the body.

What is happening is described in this commit:
getsentry/sentry-python#1573 (comment)

My question now is:
Why is BaseHTTPMiddleware not using the starlette Request object that it has to read the body but uses anyio directly?

Using the Starlette Request would cache the read data from the body stream and would make it possible to read it multiple times.

1: getsentry/sentry-python#1573

[antonpirker]

maybe @Kludex or @tomchristie can help me understand?

[antonpirker]

Just found this discussion that talks about the limitation sof BaseHTTPMiddleware, so I guess I will find my answer there:
#1729