Custom stop token

[fakerybakery]

hi,
how do i use a custom stopping sequence without using the chat class using context.evaluate?
thanks!

[giladgd]

In version 3 beta you can generate a completion using LlamaCompletion and configure stopGenerationTriggers.

For example:

import {fileURLToPath} from "url";
import path from "path";
import {getLlama, LlamaCompletion} from "node-llama-cpp";

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: path.join(__dirname, "models", "stable-code-3b.Q5_K_M.gguf")
});
const context = await model.createContext();
const completion = new LlamaCompletion({
    contextSequence: context.getSequence(),
    stopGenerationTriggers: [
        [ "];" ]
    ]
});

const input = "const arrayFromOneToTwenty = [1, 2, 3,";
console.log("Input: " + input);

const res = await completion.generateCompletion(input);
console.log("Completion: " + res);

Relevant for the 3.0.0-beta.15 version