Implement and Document LiteLLM Callbacks

[MarkRx]

LiteLLM supports callbacks to a number of different providers out of the box. These can be used to send prompt data to monitoring tools to track prompt usage and cost.

This is a stub to add support for setting the callbacks and documenting how it can be done.

I will implement this. From my preliminary POC'ing it looks like this can be done in a tool agnostic way since LiteLLM abstracts it away and pulls configuration from environment variables.

[KennyDizi]

That would be a great feature.

[mrT23]

@MarkRx
Documentation updates are always welcomed.

Regarding adding dependencies to the docker: I am, in general, careful there. I want to keep the docker small (and in fact aim to clean it up from dependencies that are not needed anymore)

Having said that, If you have tried yourself and found useful and easy-to-use one of the tools below, and are willing to share in documentation how to invoke it, we can add it to the docker, and state it as the 'recommended' pr-agent prompt logging for people who are interested. (if all the dependencies are already in litellm, even better ...)

[mrT23]

closing it as done
#1145