Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make MAX_BATCH_SIZE, MAX_NUM_TOKENS, MAX_SEQ_LENGTH user-provided input arguments #1018

Merged
merged 12 commits into from
Oct 1, 2023
Merged

Make MAX_BATCH_SIZE, MAX_NUM_TOKENS, MAX_SEQ_LENGTH user-provided input arguments #1018

merged 12 commits into from
Oct 1, 2023

Commits on Aug 22, 2023

  1. add max_tokens_per_batch, max_requests_per_batch, and max_sequence_le… 

    …ngth in RequestManager
    @jiazhihao
    jiazhihao committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    ea17767 View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Merge branch 'inference' into fix_batch_config_parameters

    @jiazhihao
    jiazhihao authored Sep 8, 2023
    Configuration menu
    Copy the full SHA
    9aab67d View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. Merge branch 'inference' into fix_batch_config_parameters

    @zwang86
    zwang86 authored Sep 29, 2023
    Configuration menu
    Copy the full SHA
    d97221e View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2023

  1. Merge branch 'inference' into fix_batch_config_parameters

    @xinhaoc
    xinhaoc authored Sep 30, 2023
    Configuration menu
    Copy the full SHA
    91ffb31 View commit details
    Browse the repository at this point in the history

  2. initial implementation

    @jiazhihao
    jiazhihao committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    b813472 View commit details
    Browse the repository at this point in the history

  3. fix c++ examples

    @jiazhihao
    jiazhihao committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    8b2523d View commit details
    Browse the repository at this point in the history

  4. fix

    @jiazhihao
    jiazhihao committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    c213942 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'inference' into fix_batch_config_parameters

    @jiazhihao
    jiazhihao authored Sep 30, 2023
    Configuration menu
    Copy the full SHA
    14d238e View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2023

  1. .

    @jiazhihao
    jiazhihao committed Oct 1, 2023
    Configuration menu
    Copy the full SHA
    432b8be View commit details
    Browse the repository at this point in the history

  2. Merge branch 'inference' of https://github.com/flexflow/FlexFlow into… 

    … fix_batch_config_parameters
    @jiazhihao
    jiazhihao committed Oct 1, 2023
    Configuration menu
    Copy the full SHA
    b3efec3 View commit details
    Browse the repository at this point in the history

  3. more tries to fix

    @jiazhihao
    jiazhihao committed Oct 1, 2023
    Configuration menu
    Copy the full SHA
    c2a4aa3 View commit details
    Browse the repository at this point in the history

  4. remove MAX_SEQ_LENGTH

    @jiazhihao
    jiazhihao committed Oct 1, 2023
    Configuration menu
    Copy the full SHA
    0aaeff4 View commit details
    Browse the repository at this point in the history