-
|
Can we pass a flag to start it? |
Beta Was this translation helpful? Give feedback.
Answered by
giladgd
Dec 11, 2023
Replies: 1 comment
-
|
The server example in An inference server is planned as part of the roadmap, but it'll take some time. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
giladgd
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The server example in
llama.cppis only an example and is not meant for production use, nor is it exposed as part of thellama.cppstable interface.It also doesn't implement significant parts of OpenAI API.
An inference server is planned as part of the roadmap, but it'll take some time.
The next version (#105) will include many improvements required to implement a sufficient API for most use cases people are using OpenAI API for.