Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tool call accuracy. #1665

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

MattZ-99
Copy link

ref: #1658.

Two updates:

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Nov 13, 2024
@shahules786
Copy link
Member

shahules786 commented Nov 13, 2024

As I just commented here I am wondering if having another metric for parallel tool call is the best option or if there is a good interface to do it inside toolcallaccuracy itself. @jjmachan @MattZ-99
Thanks a lot for the PR ❤️

@MattZ-99
Copy link
Author

Yeah. Really good point, as current ToolCallParallelAccuracy is highly inherited from the original ToolCallAccuracy.

@shahules786
Copy link
Member

Hey @MattZ-99 An update on our side, as you agreed we want to add both in one interface. ie to to toolcallaccuracy itself. But we need to find the best way for user's specify which calls should be parallel and in sequence. We will work together and make it happen, if you have any ideas please feel to share it here.

@MattZ-99
Copy link
Author

Hi @shahules786 , I have thought about it. A straightforward idea is add a parameter like ordered_tool_call which controls 'parallel' or 'sequencial' function call. What do u think about it? Or could you share your code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants