Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory optimization in async tp-linear #208

Merged
merged 29 commits into from
Aug 5, 2024
Merged

Memory optimization in async tp-linear #208

merged 29 commits into from
Aug 5, 2024

Commits on Jun 27, 2024

  1. Implemented global memory buffer to reduce activation memory of diffe… 

    …rentiable distributed operations
    @AleHD
    AleHD committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    bcf405d View commit details
    Browse the repository at this point in the history

  2. GLU fusion

    @AleHD
    AleHD committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ed1ca7d View commit details
    Browse the repository at this point in the history

  3. precommit

    @AleHD
    AleHD committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    9b0de5b View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. Merge branch 'main' into fix_tp_mem_cache

    @AleHD
    AleHD committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    bbc259f View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Wrong backward fixed

    @AleHD
    AleHD committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    803b6da View commit details
    Browse the repository at this point in the history

  2. Removed useless prints

    @AleHD
    AleHD committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    59bfb6b View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Minor fixes

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2c69e9a View commit details
    Browse the repository at this point in the history

  2. precommit

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    30439fd View commit details
    Browse the repository at this point in the history

  3. Added tp_recompute_allgather option

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1e02a9c View commit details
    Browse the repository at this point in the history

  4. Changed recompute default

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9cc81bb View commit details
    Browse the repository at this point in the history

  5. Changed recompute default

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    956fbfd View commit details
    Browse the repository at this point in the history

  6. Moved ColumnLinearNoAsync module for consistency

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b9e9201 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'fix-row-parallel' of github.com:C-TC/nanotron into asyn… 

    …c_fix
    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    cd2ff64 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'async_fix' into mem_fix_async

    @AleHD
    AleHD committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    25acc0e View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. memory efficient async linear

    @AleHD
    AleHD committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    7cc6653 View commit details
    Browse the repository at this point in the history

  2. precommit

    @AleHD
    AleHD committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    cb0f260 View commit details
    Browse the repository at this point in the history

  3. Added no_recompute_allgather mode to async

    @AleHD
    AleHD committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    6d85d03 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Merge branch 'main' into fix_tp_mem_cache

    @AleHD
    AleHD committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    49633df View commit details
    Browse the repository at this point in the history

  2. Fixed List not found

    @AleHD
    AleHD committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    2afd007 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into mem_fix_async

    @AleHD
    AleHD committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    81e7a54 View commit details
    Browse the repository at this point in the history

  4. Fixed tp=1 case

    @AleHD
    AleHD committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    7e758db View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge branch 'main' into fix_tp_mem_cache

    @AleHD
    AleHD committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ce2a96b View commit details
    Browse the repository at this point in the history

  2. Fixed column parallel

    @AleHD
    AleHD committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    cd84d4f View commit details
    Browse the repository at this point in the history

  3. Added tp_recompute_allgather test

    @AleHD
    AleHD committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    d3db06a View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix_tp_mem_cache' into mem_fix_async

    @AleHD
    AleHD committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    6f82050 View commit details
    Browse the repository at this point in the history

  5. Added tp_recompute_allgather test

    @AleHD
    AleHD committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    4c94b99 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Minor restyling

    @AleHD
    AleHD committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    7daa186 View commit details
    Browse the repository at this point in the history

  2. Fixed names

    @AleHD
    AleHD committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    31c3c5a View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1 from AleHD/fix_tp_mem_cache 

    Minor restyling
    @AleHD
    AleHD committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    0adb368 View commit details
    Browse the repository at this point in the history