Merge branch 'main' into alex/scrub-once

docs/guides/onboarding-checklist/index.md

@@ -31,6 +31,6 @@ introduction to the Logfire Web UI and show you how to interact with the data yo
 !!! note
 
     For a more comprehensive walkthrough of the Logfire Web UI and its features, you may be interested in our
-    [Logfire Web UI Guide](../web-ui/index.md).
+    [Logfire Web UI Guide](../web-ui/live.md).
 
 Let's get started! :rocket:

docs/integrations/aws-lambda.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # AWS Lambda
 
 The [`logfire.instrument_aws_lambda`][logfire.Logfire.instrument_aws_lambda] function can be used to

docs/integrations/asyncpg.md → docs/integrations/databases/asyncpg.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # asyncpg
 
 The [`logfire.instrument_asyncpg()`][logfire.Logfire.instrument_asyncpg] function can be used to instrument the [asyncpg][asyncpg] PostgreSQL driver with **Logfire**.

docs/integrations/bigquery.md → docs/integrations/databases/bigquery.md

@@ -1,3 +1,7 @@
+---
+integration: built-in
+---
+
 # BigQuery
 
 The [Google Cloud BigQuery Python client library][bigquery-pypi] is instrumented with OpenTelemetry out of the box,

docs/integrations/mysql.md → docs/integrations/databases/mysql.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # MySQL
 
 The [`logfire.instrument_mysql()`][logfire.Logfire.instrument_mysql] method can be used to instrument the [MySQL Connector/Python][mysql-connector] database driver with **Logfire**, creating a span for every query.

docs/integrations/psycopg.md → docs/integrations/databases/psycopg.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # Psycopg
 
 The [`logfire.instrument_psycopg()`][logfire.Logfire.instrument_psycopg] function can be used to instrument the [Psycopg][psycopg] PostgreSQL driver with **Logfire**. It works with both the `psycopg2` and `psycopg` (i.e. Psycopg 3) packages.

docs/integrations/pymongo.md → docs/integrations/databases/pymongo.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # PyMongo
 
 The [`logfire.instrument_pymongo()`][logfire.Logfire.instrument_pymongo] method will create a span for every operation performed using your [PyMongo][pymongo] clients.

docs/integrations/redis.md → docs/integrations/databases/redis.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # Redis
 
 The [`logfire.instrument_redis()`][logfire.Logfire.instrument_redis] method will create a span for every command executed by your [Redis][redis] clients.

docs/integrations/sqlalchemy.md → docs/integrations/databases/sqlalchemy.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # SQLAlchemy
 
 The [`logfire.instrument_sqlalchemy()`][logfire.Logfire.instrument_sqlalchemy] method will create a span for every query executed by a [SQLAlchemy][sqlalchemy] engine.

docs/integrations/sqlite3.md → docs/integrations/databases/sqlite3.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # SQLite3
 
 The [`logfire.instrument_sqlite3()`][logfire.Logfire.instrument_sqlite3] method can be used to instrument the

docs/integrations/airflow.md → docs/integrations/event-streams/airflow.md

@@ -1,3 +1,7 @@
+---
+integration: built-in
+---
+
 # Airflow
 
 **Airflow** has a native OpenTelemetry integration for [traces] and [metrics], which involves creating
@@ -158,4 +162,4 @@ otel_task_log_event = True
 [OpenTelemetry Collector]: https://opentelemetry.io/docs/collector/
 [OpenTelemetry Collector installation]: https://opentelemetry.io/docs/collector/installation/
 [OpenTelemetry Collector Receiver]: https://opentelemetry.io/docs/collector/configuration/#receivers
-[write-token]: ../guides/advanced/creating-write-tokens.md
+[write-token]: ../../guides/advanced/creating-write-tokens.md

docs/integrations/celery.md → docs/integrations/event-streams/celery.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # Celery
 
 The [`logfire.instrument_celery()`][logfire.Logfire.instrument_celery] method will create a span for every task

docs/integrations/event-streams/faststream.md

@@ -0,0 +1,50 @@
+---
+integration: built-in
+---
+
+# FastStream
+
+To instrument [FastStream][faststream] with OpenTelemetry, you need to:
+
+1. Call `logfire.configure()`.
+2. Add the needed middleware according to your broker.
+
+Let's see an example:
+
+```python title="main.py"
+from faststream import FastStream
+from faststream.redis import RedisBroker
+from faststream.redis.opentelemetry import RedisTelemetryMiddleware
+
+import logfire
+
+logfire.configure()
+
+broker = RedisBroker(middlewares=(RedisTelemetryMiddleware(),))
+
+app = FastStream(broker)
+
+
+@broker.subscriber("test-channel")
+@broker.publisher("another-channel")
+async def handle():
+    return "Hi!"
+
+
+@broker.subscriber("another-channel")
+async def handle_next(msg: str):
+    assert msg == "Hi!"
+
+
+@app.after_startup
+async def test():
+    await broker.publish("", channel="test-channel")
+```
+
+Since we are using Redis, we added the [`RedisTelemetryMiddleware`][faststream.redis.opentelemetry.RedisTelemetryMiddleware]
+to the broker. In case you use a different broker, you need to add the corresponding middleware.
+
+See more about FastStream OpenTelemetry integration in [their documentation][faststream-otel].
+
+[faststream]: https://faststream.airt.ai/latest/
+[faststream-otel]: https://faststream.airt.ai/latest/getting-started/opentelemetry/#faststream-tracing

docs/integrations/aiohttp.md → docs/integrations/http-clients/aiohttp.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # AIOHTTP Client
 
 [AIOHTTP][aiohttp] is an asynchronous HTTP client/server framework for asyncio and Python.
@@ -39,6 +43,21 @@ if __name__ == "__main__":
 
 The keyword arguments of `logfire.instrument_aiohttp_client()` are passed to the `AioHttpClientInstrumentor().instrument()` method of the OpenTelemetry aiohttp client Instrumentation package, read more about it [here][opentelemetry-aiohttp].
 
+## Hiding sensitive URL parameters
+
+The `url_filter` keyword argument can be used to modify the URL that's recorded in spans. Here's an example of how to use this to redact query parameters:
+
+```python
+from yarl import URL
+
+def mask_url(url: URL) -> str:
+    sensitive_keys = {"username", "password", "token", "api_key", "api_secret", "apikey"}
+    masked_query = {key: "*****" if key in sensitive_keys else value for key, value in url.query.items()}
+    return str(url.with_query(masked_query))
+
+logfire.instrument_aiohttp_client(url_filter=mask_url)
+```
+
 [aiohttp]: https://docs.aiohttp.org/en/stable/
 [aiohttp-server]: https://github.com/open-telemetry/opentelemetry-python-contrib/issues/501
 [opentelemetry-aiohttp]: https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/aiohttp_client/aiohttp_client.html

docs/integrations/httpx.md → docs/integrations/http-clients/httpx.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # HTTPX
 
 The [`logfire.instrument_httpx()`][logfire.Logfire.instrument_httpx] method can be used to instrument [HTTPX][httpx] with **Logfire**.

docs/integrations/requests.md → docs/integrations/http-clients/requests.md

@@ -1,3 +1,7 @@
+---
+integration: otel
+---
+
 # Requests
 
 The [`logfire.instrument_requests()`][logfire.Logfire.instrument_requests] method can be used to

docs/integrations/index.md

@@ -10,43 +10,44 @@ instrumentation package. You can find the list of all OpenTelemetry instrumentat
 
 Below you can see more details on how to use Logfire with some of the most popular Python packages.
 
-| Package                             | Type                    |
-|-------------------------------------|-------------------------|
-| [FastAPI](fastapi.md)               | Web Framework           |
-| [Django](django.md)                 | Web Framework           |
-| [Flask](flask.md)                   | Web Framework           |
-| [Starlette](starlette.md)           | Web Framework           |
-| [ASGI](asgi.md)                     | Web Framework Interface |
-| [WSGI](wsgi.md)                     | Web Framework Interface |
-| [HTTPX](httpx.md)                   | HTTP Client             |
-| [Requests](requests.md)             | HTTP Client             |
-| [AIOHTTP](aiohttp.md)               | HTTP Client             |
-| [SQLAlchemy](sqlalchemy.md)         | Databases               |
-| [Asyncpg](asyncpg.md)               | Databases               |
-| [Psycopg](psycopg.md)               | Databases               |
-| [PyMongo](pymongo.md)               | Databases               |
-| [MySQL](mysql.md)                   | Databases               |
-| [SQLite3](sqlite3.md)               | Databases               |
-| [Redis](redis.md)                   | Databases               |
-| [BigQuery](bigquery.md)             | Databases               |
-| [Airflow](airflow.md)               | Task Scheduler          |
-| [Celery](celery.md)                 | Task Queue              |
-| [Stripe](stripe.md)                 | Payment Gateway         |
-| [System Metrics](system-metrics.md) | System Metrics          |
+| Package                                  | Type                    |
+|------------------------------------------|-------------------------|
+| [FastAPI](web-frameworks/fastapi.md)     | Web Framework           |
+| [Django](web-frameworks/django.md)       | Web Framework           |
+| [Flask](web-frameworks/flask.md)         | Web Framework           |
+| [Starlette](web-frameworks/starlette.md) | Web Framework           |
+| [ASGI](web-frameworks/asgi.md)           | Web Framework Interface |
+| [WSGI](web-frameworks/wsgi.md)           | Web Framework Interface |
+| [HTTPX](http-clients/httpx.md)           | HTTP Client             |
+| [Requests](http-clients/requests.md)     | HTTP Client             |
+| [AIOHTTP](http-clients/aiohttp.md)       | HTTP Client             |
+| [SQLAlchemy](databases/sqlalchemy.md)    | Databases               |
+| [Asyncpg](databases/asyncpg.md)          | Databases               |
+| [Psycopg](databases/psycopg.md)          | Databases               |
+| [PyMongo](databases/pymongo.md)          | Databases               |
+| [MySQL](databases/mysql.md)              | Databases               |
+| [SQLite3](databases/sqlite3.md)          | Databases               |
+| [Redis](databases/redis.md)              | Databases               |
+| [BigQuery](databases/bigquery.md)        | Databases               |
+| [Airflow](event-streams/airflow.md)      | Task Scheduler          |
+| [FastStream](event-streams/faststream.md)| Task Queue              |
+| [Celery](event-streams/celery.md)        | Task Queue              |
+| [Stripe](stripe.md)                      | Payment Gateway         |
+| [System Metrics](system-metrics.md)      | System Metrics          |
 
 If you are using Logfire with a web application, we also recommend reviewing
-our [Web Frameworks](use-cases/web-frameworks.md)
+our [Web Frameworks](web-frameworks/index.md)
 documentation.
 
 ## Custom Integrations
 
 We have special integration with the Pydantic library and the OpenAI SDK:
 
-| Package                   | Type            |
-|---------------------------|-----------------|
-| [Pydantic](pydantic.md)   | Data Validation |
-| [OpenAI](openai.md)       | AI              |
-| [Anthropic](anthropic.md) | AI              |
+| Package                        | Type            |
+|--------------------------------|-----------------|
+| [Pydantic](pydantic.md)        | Data Validation |
+| [OpenAI](llms/openai.md)       | AI              |
+| [Anthropic](llms/anthropic.md) | AI              |
 
 ## Logging Integrations
 
@@ -60,3 +61,31 @@ Finally, we also have documentation for how to use Logfire with existing logging
 
 [slack]: https://join.slack.com/t/pydanticlogfire/shared_invite/zt-2b57ljub4-936siSpHANKxoY4dna7qng
 [opentelemetry]: https://opentelemetry.io/
+
+## Creating Custom Integrations
+
+If you are a maintainer of a package and would like to create an integration for **Logfire**, you can do it! :smile:
+
+We've created a shim package called `logfire-api`, which can be used to integrate your package with **Logfire**.
+
+The idea of `logfire-api` is that it doesn't have any dependencies. It's a very small package that matches the API of **Logfire**.
+We created it so that you can create an integration for **Logfire** without having to install **Logfire** itself.
+
+You can use `logfire-api` as a lightweight dependency of your own package.
+If `logfire` is installed, then `logfire-api` will use it. If not, it will use a no-op implementation.
+This way users of your package can decide whether or not they want to install `logfire`, and you don't need to
+check whether or not it's installed.
+
+Here's how you can use `logfire-api`:
+
+```python
+import logfire_api as logfire
+
+logfire.info("Hello, Logfire!")
+```
+
+!!! note
+    You generally *don't* want to call `logfire_api.configure()`, it's up to your users to call
+    `logfire.configure()` if they want to use the integration.
+
+All the **Logfire** API methods are available in `logfire-api`.

docs/integrations/anthropic.md → docs/integrations/llms/anthropic.md

@@ -1,3 +1,6 @@
+---
+integration: logfire
+---
 # Anthropic
 
 **Logfire** supports instrumenting calls to [Anthropic](https://github.com/anthropics/anthropic-sdk-python)
@@ -32,12 +35,12 @@ With that you get:
 * details of the response, including the number of tokens used
 
 <figure markdown="span">
-  ![Logfire Anthropic](../images/logfire-screenshot-anthropic.png){ width="500" }
+  ![Logfire Anthropic](../../images/logfire-screenshot-anthropic.png){ width="500" }
   <figcaption>Anthropic span and conversation</figcaption>
 </figure>
 
 <figure markdown="span">
-  ![Logfire Anthropic Arguments](../images/logfire-screenshot-anthropic-arguments.png){ width="500" }
+  ![Logfire Anthropic Arguments](../../images/logfire-screenshot-anthropic-arguments.png){ width="500" }
   <figcaption>Span arguments including response details</figcaption>
 </figure>
 
@@ -97,6 +100,6 @@ if __name__ == '__main__':
 Shows up like this in Logfire:
 
 <figure markdown="span">
-  ![Logfire Anthropic Streaming](../images/logfire-screenshot-anthropic-stream.png){ width="500" }
+  ![Logfire Anthropic Streaming](../../images/logfire-screenshot-anthropic-stream.png){ width="500" }
   <figcaption>Anthropic streaming response</figcaption>
 </figure>

docs/integrations/third-party/litellm.md → docs/integrations/llms/litellm.md

@@ -1,8 +1,11 @@
-# LiteLLM
+---
+integration: third-party
+---
 
 LiteLLM allows you to call over 100 Large Language Models (LLMs) using the same input/output format. It also supports Logfire for logging and monitoring.
 
 To integrate Logfire with LiteLLM:
+
 1. Set the `LOGFIRE_TOKEN` environment variable.
 2. Add `logfire` to the callbacks of LiteLLM.
 

docs/integrations/third-party/magentic.md → docs/integrations/llms/magentic.md

@@ -1,10 +1,14 @@
+---
+integration: third-party
+---
+
 [Magentic](https://github.com/jackmpcollins/magentic) is a lightweight library for working with
 structured output from LLMs, built around standard python type annotations and **Pydantic**. It
 integrates with **Logfire** to provide observability into prompt-templating, retries, tool/function
 call execution, and [other features](https://magentic.dev/#features).
 
 Magentic instrumentation requires no additional setup beyond configuring **Logfire** itself.
-You might also want to enable the [OpenAI](../openai.md) and/or [Anthropic](../anthropic.md) integrations.
+You might also want to enable the [OpenAI](../llms/openai.md) and/or [Anthropic](../llms/anthropic.md) integrations.
 
 ```python hl_lines="3 8 9"
 from typing import Annotated

docs/integrations/third-party/mirascope.md → docs/integrations/llms/mirascope.md

@@ -1,3 +1,7 @@
+---
+integration: third-party
+---
+
 [Mirascope][mirascope-repo] is a developer tool for building with LLMs. Their library focuses on abstractions that aren't obstructions and integrates with Logfire to make observability and monitoring for LLMs easy and seamless.
 
 You can enable it using their [`@with_logire`][mirascope-logfire] decorator, which will work with all of the [model providers that they support][mirascope-supported-providers] (e.g. OpenAI, Anthropic, Gemini, Mistral, Groq, and more).