Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batch inference support (self-assigning) #48

Open
willccbb opened this issue Jul 2, 2024 · 2 comments
Open

Batch inference support (self-assigning) #48

willccbb opened this issue Jul 2, 2024 · 2 comments
Assignees
@willccbb

Comments

@willccbb
Copy link

willccbb commented Jul 2, 2024

Would be nice to have batch inference support similar to mlx_parallm, happy to try and add soon. @Blaizzy can you assign this to me?
@Blaizzy
Copy link
Owner

Blaizzy commented Jul 3, 2024

Hey Will,

Yes, that would be awesome!

I have assigned the task to you 😀

@Blaizzy
Copy link
Owner

Blaizzy commented Jul 3, 2024

Please comment #40 here so I can assign it to you and discuss details on a single issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@willccbb willccbb

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@willccbb @Blaizzy