Model training or embedding custom data

[ZGltYQ]

I am new in LLM topic, sorry for dumb questions, can i somehow to train already pretrained model (force it to act like character) and could someone to share example how to embed documents, calendar activity, etc ?

[giladgd]

llama.cpp doesn't support training a model at the moment; you can only use existing models converted into the GGUF format with it and this library.

You can try to engineer a prompt (as long as it may be) that contains all the relevant instructions to achieve what you want on existing models without training them for your specific use case.
This is much easier in most cases, but a long prompt may take up a lot of the context, shrinking the space left for generation.

This library handles many tricks for you so you can use it with a model without engineering a long prompt by yourself to achieve what you need.
It also contains some workarounds to avoid having to tell the model what you want as part of the prompt to make it respond in a useful way.
Most of these features are only implemented in the version 3 beta for now (should be released in about a month or so, and the documentation website will be updated by then).
Right now, the documentation website is only relevant for version 2, but you can still use it to understand how this library works and get to know the basics before switching to version 3 beta.

You may be interested in using function calling to let a model dynamically fetch information and perform actions or JSON schema grammar to force a model to respond in a format you can use programmatically.
To embed documents you can use LlamaEmbeddingContext.