-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New resources for feedback loop #9607
base: main
Are you sure you want to change the base?
Conversation
…el returned from resource
}, | ||
}; | ||
|
||
const agentMessageFeedback = await AgentMessageFeedback.findAll({ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did a EXPLAIN ANALYZE for this one
Note that global agents put aside, the assistant with most feedbacks has only 9 feedbacks.
I targeted claude-3-sonnet which has 637 feedbacks across workspaces (removed the workspace filter below, to target them all and get a big number)
The query is
EXPLAIN ANALYZE
SELECT
"agent_message_feedback"."createdAt",
"agent_message_feedback"."updatedAt",
"agent_message_feedback"."agentConfigurationId",
"agent_message_feedback"."agentConfigurationVersion",
"agent_message_feedback"."thumbDirection",
"agent_message_feedback"."content",
"agent_message_feedback"."isConversationShared",
"agent_message_feedback"."id",
"agent_message_feedback"."workspaceId",
"agent_message_feedback"."agentMessageId",
"agent_message_feedback"."userId",
"agentMessage"."id" AS "agentMessage.id",
"agentMessage->message"."id" AS "agentMessage.message.id",
"agentMessage->message"."sId" AS "agentMessage.message.sId",
"agentMessage->message->conversation"."id" AS "agentMessage.message.conversation.id",
"agentMessage->message->conversation"."sId" AS "agentMessage.message.conversation.sId",
"user"."id" AS "user.id",
"user"."name" AS "user.name",
"user"."imageUrl" AS "user.imageUrl",
"user"."email" AS "user.email"
FROM
"agent_message_feedbacks" AS "agent_message_feedback"
LEFT OUTER JOIN "agent_messages" AS "agentMessage" ON "agent_message_feedback"."agentMessageId" = "agentMessage"."id"
LEFT OUTER JOIN "messages" AS "agentMessage->message" ON "agentMessage"."id" = "agentMessage->message"."agentMessageId"
LEFT OUTER JOIN "conversations" AS "agentMessage->message->conversation" ON "agentMessage->message"."conversationId" = "agentMessage->message->conversation"."id"
LEFT OUTER JOIN "users" AS "user" ON "agent_message_feedback"."userId" = "user"."id"
WHERE "agent_message_feedback"."workspaceId" = 1 AND "agent_message_feedback"."agentConfigurationId" = 'claude-3-sonnet'
ORDER BY "agent_message_feedback"."agentConfigurationVersion" DESC, "agent_message_feedback"."createdAt" DESC
LIMIT 50
The output is
-> Index Scan using conversations_pkey on conversations "agentMessage->message->conversation" (cost=0.43..0.47 rows=1 width=19) (actual time=0.003..0.003 rows=1 loops=636)
Index Cond: (id = "agentMessage->message"."conversationId")
-> Memoize (cost=0.30..5.07 rows=1 width=160) (actual time=0.001..0.001 rows=1 loops=636)
Cache Key: agent_message_feedback."userId"
Cache Mode: logical
Hits: 559 Misses: 77 Evictions: 0 Overflows: 0 Memory Usage: 24kB
-> Index Scan using users_pkey on users "user" (cost=0.29..5.06 rows=1 width=160) (actual time=0.004..0.004 rows=1 loops=77)
Index Cond: (id = agent_message_feedback."userId")
Planning Time: 0.872 ms
Execution Time: 8.725 ms
Description
Smaller version of #9368, only focused on API and resources
-> AgentMessageFeedbackResource handles sequelize basic db operations
-> front/lib.api.assistants/feedback manages extra logic around feedback creation (esp. around configurationID for global models)
-> Added a few non-attributes to model to allow sequelize joins
Risk
Break feedbacks
Deploy Plan
Deploy front, no migration needed.