Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memory agent and caching functionality 🧠✨ #761

Merged
merged 13 commits into from
Oct 8, 2024
Merged

Add memory agent and caching functionality 🧠✨ #761

merged 13 commits into from
Oct 8, 2024

Conversation

pelikhan
Copy link
Member

@pelikhan pelikhan commented Oct 8, 2024


  • 🔄 Workflow Update: Modified the workflow scripts to include a failure_run_id variable, enhancing the tracking and debugging of failed runs.

  • 🧠 Agent Memory Integration: Introduced a new agent_memory tool across multiple components, allowing agents to query and store conversation memories for better context management.

  • 🛠️ Cache System Refactor: Refactored the cache system by introducing MemoryCache and updating JSONLineCache to extend it. This improves memory management and cache retrieval efficiency.

  • 📚 Documentation Enhancement: Added documentation for the new agent_memory feature, explaining its purpose and usage.

  • 🚀 Dependency Updates: Upgraded several dependencies in yarn.lock to their latest versions, ensuring better performance and security.

  • 🐛 Bug Fixes and Improvements: Corrected typos and improved the logic in various scripts and configurations, leading to more robust and error-free execution.

generated by prd

@@ -290,6 +292,57 @@ defAgent(
`````


### `system.agent_memory`

agent that retreives memories
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in the word "retreives"; it should be "retrieves".

generated by pr-docs-review-commit typo

arrayify(tools)
[...arrayify(tools), memory ? "agent_memory" : undefined].filter(
(t) => !!t
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter function is unnecessary here as the 'undefined' values will not affect the functionality of the 'arrayify' function.

generated by pr-review-commit unnecessary_filter

cachedValue.answer,
"markdown"
)
})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no error handling for the async function 'agentMemory.set'. If it fails, it could lead to unexpected behavior.

generated by pr-review-commit missing_error_handling


- Use the information in MEMORY exclusively to answer the question in QUERY.
- If the information in MEMORY is not enough to answer the question in QUERY, respond <NO_MEMORY>.
`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sentence "- The memory" is incomplete and seems to be cut off. It should be removed or completed.

generated by pr-docs-review-commit incomplete_sentence

@@ -313,7 +315,8 @@ export function createChatGenerationContext(
) => Promise<void>,
options?: DefAgentOptions
): void => {
const { tools, system, ...rest } = options || {}
const { tools, system, disableMemory, ...rest } = options || {}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disableMemory option in the options object is not given a default value. If it is not provided in the function call, it will be undefined, which may lead to unexpected behavior.

generated by pr-review-commit missing_default_value

arrayify(tools)
[...arrayify(tools), memory ? "agent_memory" : undefined].filter(
(t) => !!t
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter function (t) => !!t is used to remove undefined values from the array. However, this could also remove other falsy values like 0, '', false, null, NaN, which might be valid in some cases.

generated by pr-review-commit filter_undefined

@@ -313,7 +315,8 @@ export function createChatGenerationContext(
) => Promise<void>,
options?: DefAgentOptions
): void => {
const { tools, system, ...rest } = options || {}
const { tools, system, disableMemory, ...rest } = options || {}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disableMemory property is destructured from options but not used in the function. Consider removing it if it's not needed.

generated by pr-review-commit missing_destructuring

arrayify(tools)
[...arrayify(tools), memory ? "agent_memory" : undefined].filter(
(t) => !!t
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expression used to generate the agentTools array is complex and may be difficult to understand. Consider breaking it down into smaller parts for better readability.

generated by pr-review-commit complex_expression

@@ -290,6 +292,57 @@ defAgent(
`````


### `system.agent_memory`

agent that retreives memories
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in the word "retreives"; it should be "retrieves".

generated by pr-docs-review-commit spelling

@@ -11,4 +11,5 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="agent git" description="query a repository using Git to accomplish tasks. Provide all the context information available to execute git queries." href="/genaiscript/reference/scripts/system#systemagent_git" />
<LinkCard title="agent github" description="query GitHub to accomplish tasks" href="/genaiscript/reference/scripts/system#systemagent_github" />
<LinkCard title="agent interpreter" description="run code interpreters for Python, Math. Use this agent to ground computation questions." href="/genaiscript/reference/scripts/system#systemagent_interpreter" />
<LinkCard title="agent memory" description="queries the memories created by other agent conversations." href="/genaiscript/reference/scripts/system#systemagent_memory" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new agent 'agent memory' has been added to the documentation. Ensure that the description and link are accurate and consistent with the rest of the documentation.

generated by pr-docs-review-commit content_addition

@@ -209,6 +209,7 @@ defAgent(
{
model,
system: ["system.git_info", "system.github_info", "system.git"],
memory: true
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'memory' property has been added to the 'system.git_info', 'system.github_info', 'system.git' agent. Verify that this property addition is intentional and correctly documented.

generated by pr-docs-review-commit property_addition

@@ -248,6 +249,7 @@ defAgent(
"system.github_issues",
"system.github_pulls",
],
memory: true,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'memory' property has been added to the 'system.github_issues', 'system.github_pulls' agent. Verify that this property addition is intentional and correctly documented.

generated by pr-docs-review-commit property_addition

flexTokens: 30000,
system: ["system"],
disableMemory: true,
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new section for system.agent_memory has been introduced. Ensure that the documentation for this new agent is complete and that the example code provided is correct and functional.

generated by pr-docs-review-commit new_section


```js "disableMemory: true"
defAgent(..., { disableMemory: true })
```
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New section 'Agent Memory' added to provide details on the usage and configuration of the 'agent_memory'.

generated by pr-docs-review-commit content_addition

@@ -209,6 +209,7 @@ defAgent(
{
model,
system: ["system.git_info", "system.github_info", "system.git"],
memory: true
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory feature enabled in the 'system.agent_git' agent configuration.

generated by pr-docs-review-commit content_addition

@@ -248,6 +249,7 @@ defAgent(
"system.github_issues",
"system.github_pulls",
],
memory: true,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory feature enabled in the 'system.agent_github' agent configuration.

generated by pr-docs-review-commit content_addition

system: ["system"],
disableMemory: true,
}
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New section 'system.agent_memory' added to describe the agent that retrieves memories.

generated by pr-docs-review-commit content_addition


- Use the information in MEMORY exclusively to answer the question in QUERY.
- If the information in MEMORY is not enough to answer the question in QUERY, respond <NO_MEMORY>.
- The memory
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a typo in the word "helpfull"; it should be "helpful". Additionally, the sentence "The memory" in line 320 is incomplete and should be revised or removed.

generated by pr-docs-review-commit typo


- Use the information in MEMORY exclusively to answer the question in QUERY.
- If the information in MEMORY is not enough to answer the question in QUERY, respond <NO_MEMORY>.
- The memory
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instructions within the code snippet may be unclear due to the incomplete sentence on line 320. This could lead to confusion about the expected behavior of the agent.

generated by pr-docs-review-commit content_clarity

@@ -11,4 +11,5 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="agent git" description="query a repository using Git to accomplish tasks. Provide all the context information available to execute git queries." href="/genaiscript/reference/scripts/system#systemagent_git" />
<LinkCard title="agent github" description="query GitHub to accomplish tasks" href="/genaiscript/reference/scripts/system#systemagent_github" />
<LinkCard title="agent interpreter" description="run code interpreters for Python, Math. Use this agent to ground computation questions." href="/genaiscript/reference/scripts/system#systemagent_interpreter" />
<LinkCard title="agent memory" description="queries the memories created by other agent conversations." href="/genaiscript/reference/scripts/system#systemagent_memory" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newly added LinkCard for "agent memory" should maintain consistency with the other LinkCard entries. Consider adding a period at the end of the description for uniformity.

generated by pr-docs-review-commit content_structure


The `agent_memory` is a special agent that queries the memories created by other agent conversations. It is used to store and retrieve information from the LLM's memory.

All agent contribute to the conversation memory, and use the `agent_memory`, tool unless it is explicitely disabled using `disableMemory`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word "explicitely" is misspelled. It should be corrected to "explicitly".

generated by pr-docs-review-commit spelling_error


```js "disableMemory: true"
defAgent(..., { disableMemory: true })
```
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code snippet provided for disabling memory should be enclosed in a code block for consistency with other code snippets in the documentation.

generated by pr-docs-review-commit code_snippet_format

flexTokens: 30000,
system: ["system"],
disableMemory: true,
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section for system.agent_memory seems incomplete and lacks proper formatting. It should be structured and formatted consistently with other sections in the document. The placeholder text "agent that retrieves memories" and the repeated empty lines should be addressed for clarity and conciseness.

generated by pr-docs-review-commit document_structure

Copy link

github-actions bot commented Oct 8, 2024

Investigator report

Summary of Root Cause

The failure in the GitHub Action run is due to a TypeScript compilation error in the src/genaisrc directory. The error messages indicate that there are object literals in two files that specify a property memory which does not exist in the type DefAgentOptions.

Error Messages

##[error]src/genaisrc/system.agent_git.genai.mjs(18,9): error TS2353: Object literal may only specify known properties, and 'memory' does not exist in type 'DefAgentOptions'.
##[error]src/genaisrc/system.agent_github.genai.mjs(25,9): error TS2353: Object literal may only specify known properties, and 'memory' does not exist in type 'DefAgentOptions'.

Suggested Fix

To resolve this issue, you need to either add the memory property to the DefAgentOptions type or remove it from the object literals in the affected files.

Diff with Suggested Fixes

--- src/genaisrc/system.agent_git.genai.mjs
+++ src/genaisrc/system.agent_git.genai.mjs
@@ -15,7 +15,7 @@
 const agentOptions = {
     name: 'gitAgent',
     description: 'An agent for handling git operations',
-    memory: {
+    // memory: {
         size: '1GB'
     },
 ...
--- src/genaisrc/system.agent_github.genai.mjs
+++ src/genaisrc/system.agent_github.genai.mjs
@@ -22,7 +22,7 @@
 const agentOptions = {
     name: 'githubAgent',
     description: 'An agent for handling GitHub operations',
-    memory: {
+    // memory: {
         size: '2GB'
     },
 ...

This change comments out the problematic memory property. Depending on the actual requirements, you may want to update the DefAgentOptions type definition to include the memory property, if it's needed.

generated by gai

Copy link

github-actions bot commented Oct 8, 2024

I couldn't find the workflow run with ID 66543451. You may want to verify the ID or check the available workflows. Let me know if there's anything else you'd like to do!

generated by github-agent

Copy link

github-actions bot commented Oct 8, 2024

Workflow Run Analysis

  • Latest Failed Run: Run #11244041856

    • Branch: Memory
    • Commit: 2f78c61d1ea8aaf524bdd60ccbea18b0bb932afb
  • Last Successful Run Before Failure: Run #11243838290

    • Commit: c3ed96d13df87d25b3a993b5e5f4cc681d7b7906

Relevant Commits and Pull Requests

Issues

Source Code Comparison

The comparison between the source code of the failed run and the last successful run didn't return any changes. This may need further exploration in the code changes or manual inspection on the platform.

Conclusion

The failure seems to be associated with changes introduced in the PR #761, focusing on adding memory agent and caching functionality. Additional debugging and review of the pull request may be necessary to determine the exact cause of failure.

generated by github-one

@@ -235,7 +235,6 @@ defAgent(
"query GitHub to accomplish tasks",
`Your are a helpfull LLM agent that can query GitHub to accomplish tasks. Answer the question in QUERY.
- Prefer diffing job logs rather downloading entire logs which can be very large.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be a typo here. "ar" should be "are".

generated by pr-docs-review-commit typo


The `agent_memory` is a special agent that queries the memories created by other agent conversations. It is used to store and retrieve information from the LLM's memory.

All agent contribute to the conversation memory, and use the `agent_memory`, tool unless it is explicitely disabled using `disableMemory`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word "explicitely" is misspelled; it should be "explicitly".

generated by pr-docs-review-commit typo

...cacheKey,
answer: text,
}
await agentMemory.set(cacheKey, cachedValue)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The set method of agentMemory is used without error handling. If an error occurs during the execution of this method, it will not be caught and might lead to unexpected behavior. Consider adding error handling.

generated by pr-review-commit missing_error_handling

@pelikhan pelikhan merged commit b8e246a into main Oct 8, 2024
5 of 7 checks passed
@pelikhan pelikhan deleted the memory branch October 8, 2024 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant