Support fallback deployments to accompany fallback models

[zmeir]

This is useful for example in Azure OpenAI deployments where you have a different deployment per model, so the current fallback implementation doesn't work (still uses the same deployment for each fallback attempt)

[mrT23]

@okotek

[CodiumAI-Agent]

Question: write me a poem about this PR

Answer:
In the realm of code, where AI models play,
A pull request arrives, to brighten the day.
"Support fallback deployments," it proudly proclaims,
To enhance Azure OpenAI, and elevate its games.

No longer bound, to a single deployment id,
It adapts and adjusts, as the situation did.
A property it becomes, dynamically retrieved,
An elegant solution, cleverly conceived.

With each model a deployment, it pairs,
Ensuring each fallback, is free of snares.
If one fails, another takes the stage,
A dance choreographed, across the digital page.

In the settings template, a new field appears,
To list fallback deployments, easing our fears.
A symphony of changes, in Python's sweet song,
A pull request that rights, a deployment wrong.

So here's to the coder, who saw the need,
And crafted this PR, with impressive speed.
In the world of AI, they've made their mark,
A beacon of progress, in the coding dark.

[CodiumAI-Agent]

Question: is the PR complete? (answer as if you are Elon Musk)

Answer:
The PR appears to be complete as it successfully implements the feature of supporting fallback deployments to accompany fallback models. This is particularly useful for Azure OpenAI deployments where each model has a different deployment. The code changes include the addition of a new property deployment_id, modifications to the chat_completion and retry_with_fallback_models methods, and updates to the settings file. The PR also includes appropriate logging and error handling. However, it would be beneficial to include tests to ensure the new functionality works as expected.

[okotek]

Suggested change

	#fallback_deployments = [] # Match your fallback models from configuration.toml with the appropriate deployment_id
	#fallback_deployments = [] # For each fallback model specified in configuration.toml in the [config] section, specify the appropriate deployment_id

[zmeir]

Done. Thanks for the clarification :)

[okotek]

/improve