Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New resources for feedback loop #9607

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

overmode
Copy link
Contributor

@overmode overmode commented Dec 23, 2024

Description

Smaller version of #9368, only focused on API and resources
-> AgentMessageFeedbackResource handles sequelize basic db operations
-> front/lib.api.assistants/feedback manages extra logic around feedback creation (esp. around configurationID for global models)
-> Added a few non-attributes to model to allow sequelize joins

Risk

Break feedbacks

Deploy Plan

Deploy front, no migration needed.

Copy link

github-actions bot commented Dec 23, 2024

Warnings
⚠️

Files in **/lib/models/ have been modified and the PR has the migration-ack label. Don't forget to run the migration from prodbox.

Generated by 🚫 dangerJS against abd155a

@overmode overmode added the migration-ack 📁 Label to acknowledge that a migration is required. label Dec 23, 2024
},
};

const agentMessageFeedback = await AgentMessageFeedback.findAll({
Copy link
Contributor Author

@overmode overmode Dec 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a EXPLAIN ANALYZE for this one
Note that global agents put aside, the assistant with most feedbacks has only 9 feedbacks.

I targeted claude-3-sonnet which has 637 feedbacks across workspaces (removed the workspace filter below, to target them all and get a big number)

The query is

EXPLAIN ANALYZE
SELECT 
"agent_message_feedback"."createdAt", 
"agent_message_feedback"."updatedAt", 
"agent_message_feedback"."agentConfigurationId", 
"agent_message_feedback"."agentConfigurationVersion", 
"agent_message_feedback"."thumbDirection", 
"agent_message_feedback"."content", 
"agent_message_feedback"."isConversationShared", 
"agent_message_feedback"."id", 
"agent_message_feedback"."workspaceId", 
"agent_message_feedback"."agentMessageId", 
"agent_message_feedback"."userId", 
"agentMessage"."id" AS "agentMessage.id", 
"agentMessage->message"."id" AS "agentMessage.message.id", 
"agentMessage->message"."sId" AS "agentMessage.message.sId", 
"agentMessage->message->conversation"."id" AS "agentMessage.message.conversation.id", 
"agentMessage->message->conversation"."sId" AS "agentMessage.message.conversation.sId", 
"user"."id" AS "user.id", 
"user"."name" AS "user.name", 
"user"."imageUrl" AS "user.imageUrl", 
"user"."email" AS "user.email" 
FROM 
"agent_message_feedbacks" AS "agent_message_feedback" 
LEFT OUTER JOIN "agent_messages" AS "agentMessage" ON "agent_message_feedback"."agentMessageId" = "agentMessage"."id" 
LEFT OUTER JOIN "messages" AS "agentMessage->message" ON "agentMessage"."id" = "agentMessage->message"."agentMessageId" 
LEFT OUTER JOIN "conversations" AS "agentMessage->message->conversation" ON "agentMessage->message"."conversationId" = "agentMessage->message->conversation"."id" 
LEFT OUTER JOIN "users" AS "user" ON "agent_message_feedback"."userId" = "user"."id" 
WHERE  "agent_message_feedback"."workspaceId" = 1 AND "agent_message_feedback"."agentConfigurationId" = 'claude-3-sonnet' 
ORDER BY "agent_message_feedback"."agentConfigurationVersion" DESC, "agent_message_feedback"."createdAt" DESC 
LIMIT 50

The output is

-> Index Scan using conversations_pkey on conversations "agentMessage->message->conversation" (cost=0.43..0.47 rows=1 width=19) (actual time=0.003..0.003 rows=1 loops=636)
Index Cond: (id = "agentMessage->message"."conversationId")
-> Memoize (cost=0.30..5.07 rows=1 width=160) (actual time=0.001..0.001 rows=1 loops=636)
Cache Key: agent_message_feedback."userId"
Cache Mode: logical
Hits: 559 Misses: 77 Evictions: 0 Overflows: 0 Memory Usage: 24kB
-> Index Scan using users_pkey on users "user" (cost=0.29..5.06 rows=1 width=160) (actual time=0.004..0.004 rows=1 loops=77)
Index Cond: (id = agent_message_feedback."userId")
Planning Time: 0.872 ms
Execution Time: 8.725 ms

@overmode overmode marked this pull request as ready for review December 24, 2024 15:07
@overmode overmode requested a review from Fraggle December 24, 2024 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
migration-ack 📁 Label to acknowledge that a migration is required.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant