Memgpt / Memory Management

[scenaristeur]

I would like to implement à memgpt like memory management https://memgpt.readme.io/docs/test_page

• when context is too long, using a shiftContext strategy to store some parts of context in a externat support (file or vectordatabase) and keeping référence to that shift in the original context.

I've looked at

node-llama-cpp/src/evaluator/LlamaChat/utils/contextShiftStrategies/eraseFirstResponseAndKeepFirstSystemChatContextShiftStrategy.ts

Line 7 in 51eab61

export async function eraseFirstResponseAndKeepFirstSystemChatContextShiftStrategy({

using llama3.1ChatWrapper, but i'm not able to réinject the new correct context.

I'm using this kind of exemple whit a httpRequest / Get function https://github.com/scenaristeur/dady/blob/main/llm/node_llama/node_llama_cpp_3/memWrapper.js
I can ask "what does the page https://my-website talk about?".
I see the function call working properly, but thé result for a long page overlaps thé context size.
Thé goal is to use thé contextshift, to store thé result in some other memory, to be able to accès it later. But when i m not able to réinject thé New context. Did anyone succed in a shiftContext opération ? What is thé way to do that ? Can someone provide à working example ? (Erasefirst... does not seem to work for tgis example).

How can i track the content of the context used / available ?
Can the model / the session be aware of how much of the context is used and decide to condense / resume and save important infos like https://memgpt.readme.io/docs/test_page

Here is thé memgpt agent https://github.com/cpacker/MemGPT/blob/main/letta/agent.py

[giladgd]

If I understand what you're trying to do correctly, you want to analyze the model response and summarize key points to add them as given information as part of the system prompt.
You may want to do that after each prompt, or only when a context shift happens.

To do that, after each prompt you can get the chat history state and rewrite it to make it more concise and update the key points in the system prompt.
Here's an example or how you can get the current chat state and set a new state.
Just modify the chat history state and set it again after you get the response from a prompt.

Regarding the context shift, the function that you referenced is responsible for a context shift strategy that attempts to delete old messages to make room for new ones, without removing the system prompt and other prioritized information, to make enough room for new token to be generated.
You can customize it by passing a custom strategy function that returns a new chat history. That function can even utilize another context sequence or even a different model to analyze the chat history to compact it.

Note that a context shift happens only when the context window is full.

To find out what's the current information that's loaded to the context sequence state, you can call getLastEvaluationContextWindow on a chat session.

To check how many tokens are in use in the current context sequence state, you can check nextTokenIndex, and you can even check out contextTokens to see their values.