[Error: could not find a KV slot for the batch (try reducing the size of the batch or increase the context)]

[juned-adenwalla]

I am facing a critical issue in my application, which is causing a major blocker. The error message I'm encountering is: "Error: could not find a KV slot for the batch (try reducing the size of the batch or increase the context)."

I am sharing the code snippet I am using below. Could you please guide me on how to resolve this issue?

import path from 'path';
import { LlamaModel, LlamaContext, LlamaChatSession } from "node-llama-cpp";

const __dirname = path.resolve();

const modelName = 'phi-2-orange.Q2_K.gguf';
const modelURL = 'https://huggingface.co/TheBloke/phi-2-orange-GGUF/resolve/main/phi-2-orange.Q2_K.gguf';
const modelsFolder = path.join(__dirname, 'models');
const modelFilePath = path.join(modelsFolder, modelName);

const model = new LlamaModel({ modelPath: modelFilePath, useMlock: true });
const context = new LlamaContext({ model });
const session = new LlamaChatSession({ context });

async function askQuestion(question) {
    console.log("User: " + question);
    try {
        const answer = await session.prompt(question);
        console.log("AI: " + answer);
    } catch (error) {
        console.error("Error:", error);
    }
}

async function main() {
    await session.init();
    await askQuestion("Hi there, how are you?");
    await askQuestion("Summarize what you said");
}

main();

[giladgd]

This issue was already solved in version 3 beta

[juned-adenwalla]

Heloo thank you for responding I did use the beta version but still after sending prompt is blank with no error now

[juned-adenwalla]

Hii any update please its been 3 days now I am stuck

[giladgd]

Please provide a complete reproducible example using the version 3 beta I can use to investigate.
There aren't enough details here for me to be able to help you.

Please include details about your environment: OS type, OS arch, OS version, nodejs version, node-llama-cpp version.
Also, include the output of this command: npx node-llama-cpp@beta inspect gpu