-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: refactor out ContextWindow
and ChatLogs
from the Chat
class
#244
Conversation
msg: ChatMessage | ChatMessage[] | ||
) { | ||
const msgLog = this._msgLogs[type]; | ||
let msgs = Array.isArray(msg) ? msg : [msg]; | ||
|
||
for (const msg of msgs) { | ||
const tokens = this._tokenizer.encode(msg.content || "").length; | ||
|
||
// If the message is larger than the context window |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the comment strictly applies now, we could delete it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No it's still accurate. If a context window is present, we want to give it a chance to "chunk" a message that is above it's desired size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm being a nit, so feel free to ignore, but the message refers to "if the message is larger..." and the functionality (this._contextWindow?.shouldChunk(tokens)) {
has no concept of "larger" (it's an internal implementation detail)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah okay, I see what you mean. So a better comment would be something like "See if the context window needs to chunk the message"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, exactly.
Or just removing the comment, since it's already evident from the code.
For the most part (99%) I really only believe in comments that explain "why" the code does what it does instead of "what" the code does (cause the latter should always be achievable through cleaner code)
The
Chat
class was not very easy to use throughout the code. In order to:I've created the
ContextWindow
andChatLogs
classes, which simply contain logic that was previously bundled into theChat
class.Usage:
Motivation: I've been wanting to polish the Chat typings for some time as I think the current implementation does not separate concerns well. This leads to problems like this, where we've regressed and are no longer summarizing the
toSummarize
messages here https://github.com/polywrap/evo.ninja/blob/962da61051760aad97a2b93d909ccdc9310c9f34/packages/agent-utils/src/llm/Chat.ts#L212