Support for AWS Bedrock

[abtawfik]

It would be nice to have support for Bedrock so that application that use AWS can integrate more seamlessly.

[samuelcolvin]

PR welcome to add AWS Bedrock.

I think from a brief search that Bedrock is like VertexAI - you use a custom API, but the format of the response matches the foundational model used, so actually using Bedrock with (say) Anthropic is more about customising the URL and auth, then using the (as yet unimplemented, see #63) Anthropic logic.

That's how The VertexAIModel works - it does it's own auth and sets the url, then just returns an instance of GeminiAgentModel.

[stephenhibbert]

@samuelcolvin - a couple of points to add...

The Anthropic SDK currently supports Amazon Bedrock though the InvokeModel API with a HTTPX client and boto3 for auth.

Amazon Bedrock also has a newer Converse API that provides a consistent interface for all the models it supports. This includes the new Amazon Nova models as well as models from Meta, Mistral and Cohere models, the full list is here.

Maybe it makes sense to add AnthropicBedrock support in the Anthropic PR (since I see you're using the anthropic python SDK) and then handle the broader Bedrock support separately with the Converse API?

[PabloSimonSainz]

Hi, I can help with this issue

[YanSte]

Any help needed ? I can work on it. @PabloSimonSainz or @samuelcolvin

[dmenini]

Following up on what @stephenhibbert said, currently the AnthropicModel class accepts the anthropic_client param with type AsyncAnthropic. However, the current logic also works with the AsyncAnthropicBedrock client from anthropic.

I agree that it would be nice to have built-in support for any model on Bedrock, but it's still a valid workaround for those who are using Anthropic models :) Although it would be nice to have the correct type hint to take both clients into account.

[stephenhibbert]

Spot on @dmenini. As a workaround we can install boto3 and pass an instance of AsyncAnthropicBedrock to the AnthropicModel like this

from anthropic import AsyncAnthropicBedrock
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai import Agent
from pydantic import BaseModel

anthropic_bedrock_client = AsyncAnthropicBedrock(aws_region='us-east-1')

model = AnthropicModel(
    model_name='anthropic.claude-3-haiku-20240307-v1:0',
    anthropic_client=anthropic_bedrock_client
)

class MyModel(BaseModel):
    city: str
    country: str

print(f'Using model: {model}')
agent = Agent(model, result_type=MyModel)

if __name__ == '__main__':
    result = agent.run_sync('The windy city in the US of A.')
    print(result.data)
    print(result.usage())


Using model: AnthropicModel(model_name='anthropic.claude-3-haiku-20240307-v1:0')    
city='Chicago' country='United States'
Usage(requests=1, request_tokens=459, response_tokens=56, total_tokens=515, details=None)

Process finished with exit code 0   

But it would be nice to have 1) documentation for this in the Anthropic section 2) correct type hints for Anthropic Bedrock models 3) full Bedrock support for any model using the ConverseAPI. Happy to help work on/review/test this too.

This pydantic/logfire#701 should also mean the logfire integration will work out the box when merged.

[PierreColombo]

Hey,
I'm still interrested in pydantic/logfire#701 to be merged :) this will remove some friction to adapt my current code!