Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom AllReduce #1467

Merged
merged 54 commits into from
Aug 15, 2024
Merged

Custom AllReduce #1467

merged 54 commits into from
Aug 15, 2024

Commits on Jun 29, 2024

  1. feat: update deps/flashinfer

    @chenzhuofu
    chenzhuofu committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    7e02ef0 View commit details
    Browse the repository at this point in the history

  2. feat: update flashinfer

    @chenzhuofu
    chenzhuofu committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    be64b3b View commit details
    Browse the repository at this point in the history

  3. fix: now can get correct result, but has performance problem

    @chenzhuofu
    chenzhuofu committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    7a4543b View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. fix: update_custom_mask performance

    @chenzhuofu
    chenzhuofu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    e1c8145 View commit details
    Browse the repository at this point in the history

  2. chore: minor

    @chenzhuofu
    chenzhuofu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    483b71b View commit details
    Browse the repository at this point in the history

  3. chore: add perf code

    @chenzhuofu
    chenzhuofu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    0a117fb View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Merge branch 'specscheduler' into specscheduler-new-attention

    @zikun-li
    zikun-li committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    ac343ee View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. feat: add attention metadata

    @chenzhuofu
    chenzhuofu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3e9dc00 View commit details
    Browse the repository at this point in the history

  2. feat: add AttentionMetaData

    @chenzhuofu
    chenzhuofu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a498e6f View commit details
    Browse the repository at this point in the history

  3. feat: tree_verify_attn use global attentionmetadata

    @chenzhuofu
    chenzhuofu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    2903996 View commit details
    Browse the repository at this point in the history

  4. feat: move attentionmetasize to global computing

    @chenzhuofu
    chenzhuofu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    fed9e5c View commit details
    Browse the repository at this point in the history

  5. chore: minor

    @chenzhuofu
    chenzhuofu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7c64f36 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. chore: remove unused

    @chenzhuofu
    chenzhuofu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    37d3e3d View commit details
    Browse the repository at this point in the history

  2. feat: add spec_inc_attn backup

    @chenzhuofu
    chenzhuofu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    9940ff5 View commit details
    Browse the repository at this point in the history

  3. feat: SSM use flashinfer kernel

    @chenzhuofu
    chenzhuofu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    5262af7 View commit details
    Browse the repository at this point in the history

  4. fix: SSM don't use cudaGraph

    @chenzhuofu
    chenzhuofu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    dfe4bec View commit details
    Browse the repository at this point in the history

  5. chore: remove redundant code

    @chenzhuofu
    chenzhuofu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    e08f06e View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2024

  1. chore: comment out minor

    @chenzhuofu
    chenzhuofu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    ed544e9 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. feat: attention adapt to cudaGraph

    @chenzhuofu
    chenzhuofu committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    214aed6 View commit details
    Browse the repository at this point in the history

  2. fix: split handler_collections for prompt/decode phases

    @chenzhuofu
    chenzhuofu committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    020129b View commit details
    Browse the repository at this point in the history

  3. chore: tree verify cannot use cudaGraph

    @chenzhuofu
    chenzhuofu committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0150501 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. feat: move all flashinfer-related states to global (tree search atten… 

    …tion)
    @chenzhuofu
    chenzhuofu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    d4d66af View commit details
    Browse the repository at this point in the history

  2. fix: use identical attention_meta instance across all FFHandlers

    @chenzhuofu
    chenzhuofu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    b4cc53b View commit details
    Browse the repository at this point in the history

  3. feat: enable cudaGraph in tree search mode

    @chenzhuofu
    chenzhuofu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    79ea1d9 View commit details
    Browse the repository at this point in the history

  4. chore: minor

    @chenzhuofu
    chenzhuofu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    8d52e4f View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. feat: tree search & verify use separate attention_meta

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2afb66c View commit details
    Browse the repository at this point in the history

  2. fix: attention_metadata should be distinct for each worker

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    74e166f View commit details
    Browse the repository at this point in the history

  3. feat: tree verify attention use metadata

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    252a5c4 View commit details
    Browse the repository at this point in the history

  4. feat: support llm cudaGraph

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    9fda3b6 View commit details
    Browse the repository at this point in the history

  5. chore: minor

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    6f88134 View commit details
    Browse the repository at this point in the history

  6. chore: temporally only enable SSM cudaGraph for performance issue

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    dca40b8 View commit details
    Browse the repository at this point in the history

  7. chore: minor

    @chenzhuofu
    chenzhuofu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    0e5ec41 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2024

  1. fix: llm cudaGraph, should ensure the kernel parameter be consistent

    @chenzhuofu
    chenzhuofu committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    dd4d6d0 View commit details
    Browse the repository at this point in the history

  2. feat: reduce cudaGraph number

    @chenzhuofu
    chenzhuofu committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    18fee1c View commit details
    Browse the repository at this point in the history

  3. feat: reduce cudaGraph instances number

    @chenzhuofu
    chenzhuofu committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    b17c5cb View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. feat: add tensorRT-LLM custom_allreduce

    @chenzhuofu
    chenzhuofu committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    428875c View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. feat: add tensorrt_llm custom_allreduce kernel into exeutable

    @chenzhuofu
    chenzhuofu committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    840da50 View commit details
    Browse the repository at this point in the history

  2. doc: add a README for acknowledgement

    @chenzhuofu
    chenzhuofu committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    5f16bd4 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. feat: add device info in FFHandle

    @chenzhuofu
    chenzhuofu committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    3d50053 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. feat: enable both cudaGraph

    @chenzhuofu
    chenzhuofu committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    273dfc7 View commit details
    Browse the repository at this point in the history

  2. feat: temporally add the ipc mem

    @chenzhuofu
    chenzhuofu committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    d201b2e View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. feat: enable only ssm cudaGraph

    @chenzhuofu
    chenzhuofu committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    1ba38a8 View commit details
    Browse the repository at this point in the history

  2. feat: minor reconstruct

    @chenzhuofu
    chenzhuofu committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    7b235ef View commit details
    Browse the repository at this point in the history

  3. feat: implementation of CommunicationBuffer

    @chenzhuofu
    chenzhuofu committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    f3c9629 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. feat: implement custom_allreduce

    @chenzhuofu
    chenzhuofu committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    462e0b7 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. feat: allocate memory from legion, not cudaMalloc

    @chenzhuofu
    chenzhuofu committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    567e165 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. chore: some debug output

    @chenzhuofu
    chenzhuofu committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    a2fb367 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. feat: switch to use peer memory, rather than IPC memory

    @chenzhuofu
    chenzhuofu committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4134013 View commit details
    Browse the repository at this point in the history

  2. chore: remove debug output

    @chenzhuofu
    chenzhuofu committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    c094981 View commit details
    Browse the repository at this point in the history

  3. fix: minor concurrent bug

    @chenzhuofu
    chenzhuofu committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    95ca71c View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. Merge branch 'specscheduler' of github.com:flexflow/FlexFlow into spe… 

    …cscheduler-new-attention
    @chenzhuofu
    chenzhuofu committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    55a7942 View commit details
    Browse the repository at this point in the history

  2. style: format code

    @chenzhuofu
    chenzhuofu committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b9760c0 View commit details
    Browse the repository at this point in the history

  3. chore: remove unused backup code

    @chenzhuofu
    chenzhuofu committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    102dd38 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. chore: more measurements

    @chenzhuofu
    chenzhuofu committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    0219b78 View commit details
    Browse the repository at this point in the history