Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add Converse API, add LLama 3.2 #177

Open
wants to merge 14 commits into
base: development
Choose a base branch
from
Open

feat: add Converse API, add LLama 3.2 #177

wants to merge 14 commits into from
+1,612 −583

Conversation

roman-romanov-o
Copy link
Contributor

@roman-romanov-o roman-romanov-o commented Nov 8, 2024 •
edited
Loading

  • Added Converse API

  • Removed Llama 2 (end of life in Bedrock)

  • Add Llama 3.2 (1B, 3B, 11B, 90B)

  • Tool support for Llama 3.1 70B, Llama 3.1 405B, Llama 3.2 90B

  • Move all Llama models to Converse API

roman-romanov-o
roman-romanov-o commented Nov 8, 2024
View reviewed changes
tests/utils/openai.py Outdated Show resolved Hide resolved
@roman-romanov-o
Copy link
Contributor Author

roman-romanov-o commented Nov 8, 2024 •
edited by ai-dial-actions
Loading

/deploy-review

Environment URL: https://chat-ai-dial-adapter-bedrock-pr-177.nightly-test.deltixhub.io
E2E tests status: success

adubovik
adubovik reviewed Nov 8, 2024
View reviewed changes
README.md Outdated Show resolved Hide resolved
@roman-romanov-o
Copy link
Contributor Author

roman-romanov-o commented Nov 15, 2024 •
edited by ai-dial-actions
Loading

/deploy-review

Environment URL: https://chat-ai-dial-adapter-bedrock-pr-177.nightly-test.deltixhub.io
E2E tests status: success

adubovik
adubovik reviewed Nov 18, 2024
View reviewed changes
aidial_adapter_bedrock/llm/converse/adapter.py Outdated
params: ModelParameters,
) -> ConverseRequestWrapper:
system_prompt_extraction = extract_converse_system_prompt(messages)
processed_messages = await to_converse_messages(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

converse_messages = ...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

adubovik
adubovik reviewed Nov 18, 2024
View reviewed changes
aidial_adapter_bedrock/llm/converse/input.py Outdated
for idx, msg in enumerate(messages)
]

# Merge messages with same roles, to preserve turn-based user/assistant turns
Copy link
Collaborator

@adubovik adubovik Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# Merge messages with the same roles to achieve an alternation of user-assistant roles.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

adubovik
adubovik reviewed Nov 18, 2024
View reviewed changes
aidial_adapter_bedrock/llm/converse/input.py Outdated

return {
"role": msg1["role"],
"content": list(content1) + list(content2),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"content": content1 + content2,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

adubovik
adubovik reviewed Nov 18, 2024
View reviewed changes
aidial_adapter_bedrock/llm/converse/input.py Outdated
"content": list(content1) + list(content2),
}, set1 | set2

converted: List[Tuple[ConverseMessage, Set[int]]] = [
Copy link
Collaborator

@adubovik adubovik Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    converted = [
        (await to_converse_message(msg, storage), {idx})
        for idx, msg in enumerate(messages, start=start_offset)
    ]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adubovik adubovik adubovik left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@roman-romanov-o @adubovik