Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split prefilling batch with decoding batch for increamental decoding. #1345

Closed
wants to merge 4 commits into from
Closed

Split prefilling batch with decoding batch for increamental decoding. #1345

wants to merge 4 commits into from

Conversation

zwang86
Copy link
Collaborator

@zwang86 zwang86 commented Mar 29, 2024 •
edited by wmdi
Loading

Description of changes:

Related Issues:

Linked Issues:

  • Issue #

Issues closed by this PR:

  • Closes #

This change is Reviewable

@jiazhihao jiazhihao added the inference Features and fixes related to the inference project. label May 23, 2024
@jiazhihao
Copy link
Collaborator

@zwang86 @zikun-li Has this already been merged to the spec-scheduler branch?

@zwang86
Copy link
Collaborator Author

zwang86 commented May 31, 2024

@jiazhihao This pr is out of date, we can close this now.

@zwang86 zwang86 closed this May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
inference Features and fixes related to the inference project.
Projects
FlexFlow Serve
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zwang86 @jiazhihao