Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable usage token and use Converse API with all Bedrock models #569

Merged
merged 4 commits into from
Sep 19, 2024

Conversation

charles-marion
Copy link
Collaborator

Issue #, if available:
Follow up of #564

Description of changes:
Since the bedrock converse api allows a team to "write code once and use it with different models.", this change aims to use the same code for all Bedrock models.

The only differences are some models do not support system prompt or streaming.

The main goal of this change is to add usage token tracking to all Bedrock models.

Changes

  • Remove model specific code for Bedrock
  • Add unit test for the bedrock adapters
  • Add an user friendly message is a bedrock model is not enabled
  • Add unit test

Notes

  • the main changes are in lib/model-interfaces/langchain/functions/request-handler/adapters/bedrock/base.py. Github hides this file by default
  • since all the models use the same prompts. LLama custom prompts are no longer used and it relies on the Converse API.

Testing
Called the models one by and checked the usage

image

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@charles-marion charles-marion changed the title feat: Add usage token and use Converse API for Bedrock models feat: Enable usage token and use Converse API with all Bedrock models Sep 17, 2024
@charles-marion charles-marion merged commit e0d310e into aws-samples:main Sep 19, 2024
1 check passed
@charles-marion charles-marion deleted the converse_api branch September 19, 2024 15:00
lloydclowes pushed a commit to lloydclowes/gen-ai-playground that referenced this pull request Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants