Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Draft] To get reasonable performance on the default Triton passes pipeline. #897

Closed
wants to merge 19 commits into from
Closed

[Draft] To get reasonable performance on the default Triton passes pipeline. #897

wants to merge 19 commits into from

Conversation

chengjunlu
Copy link
Contributor

@chengjunlu chengjunlu commented Apr 17, 2024 •
edited
Loading

Draft only for CI.

Added:

  • Intel rewriter tensor pointer pass.
  • Intel remove layout pass.
  • Intel accelerate matmul pass.
  • Intel materialize 2D load pass.
  • Intel loop pipelining with prefetching.
  • Use the intel specific passes pipeline.
  • Use large 2D load
  • Lower the prefetching op to 2D prefetching op.

TODO:

  • Benchmark.
  • Use the sub-group-size=32.
  • Use the 2D store
  • Add double GRF as a compile configuration.
  • Change the convert layout and emit index to dense stride for the dot operands layout and DPAS layout,

Memo: The sub-group-size=32 cause some DPAS UT fail.
Memo: Need to upstream the nested layout of the dot operand layout. Slice of dot layout. dot of dot layout.

@chengjunlu chengjunlu linked an issue Apr 17, 2024 that may be closed by this pull request
[Performance] Enhance the Triton GEMM/Flash attention kernel performance for the default Triton passes pipeline #878
Closed
@chengjunlu chengjunlu force-pushed the chengjun/llvm-target-dev branch 13 times, most recently from 759e4e1 to d3c482b Compare April 24, 2024 06:05
@chengjunlu chengjunlu force-pushed the chengjun/llvm-target-dev branch 4 times, most recently from 75ed1be to ec24ea5 Compare April 29, 2024 08:48
@chengjunlu chengjunlu force-pushed the chengjun/llvm-target-dev branch 4 times, most recently from 7b22c45 to 0c25d6a Compare May 8, 2024 08:33
@whitneywhtsang whitneywhtsang marked this pull request as draft May 11, 2024 04:17
@chengjunlu chengjunlu force-pushed the chengjun/llvm-target-dev branch 2 times, most recently from 22daa3f to bd400b0 Compare May 15, 2024 06:19
@chengjunlu chengjunlu closed this May 15, 2024
chengjunlu added 4 commits May 15, 2024 10:11
@chengjunlu
benchmark codes.
61912b9
@chengjunlu
Add prefetch ops lowering pattern.
706c438
@chengjunlu
Add MaterializeBlockPointer pass to mark the tt.load operations which…
fe89a95 
… could be supported by Intel GPU hardware 2D memory accessing. To protect the block pointer from being re-write in RewriteTensorPointer pass.
@chengjunlu
Add Intel GPU 2D memory accessing for 2D load in code generation.
90e2c98
chengjunlu and others added 15 commits May 15, 2024 10:11
@chengjunlu
Add more sanity check in SharedToDotOperandDPAS.cpp.
2e2a4b3
@chengjunlu
Add Intel rewriter tensor pointer pass to rewrite all load/store sema…
8bb4220 
…ntics initiated by a `tt.make_tensor_ptr` and `tt.advance` in TTGIR.
@chengjunlu
Update the llPrintf signature.
0acd95e
@chengjunlu
Use the GenISA.
b0d2db8
@chengjunlu
Use the DPAS when threads_per_warp=32.
585bfec
@chengjunlu
Change threads_per_warp to 16.
aa40452
@chengjunlu
Update the compiler pipeline
479ef9c
@chengjunlu
Add the nested layout with dot op layout.
7609c5c 
Such as `#triton_gpu.slice<{dim = 1, parent = #triton_gpu.dot_op<{opIdx = 0, parent = #mma}>}>`
@chengjunlu
Fix issue to support fp64
c83861b
@chengjunlu
Add debug print thread info
2b6bacf
@chengjunlu
Add tensor print utils.
8bc0087
@chengjunlu
coordinate go first for compare
38dfe5b
@chengjunlu
clean up
36259ca
@victor-eds @chengjunlu
[Upstream] Add threads_per_warp field to autotuner configurations
cd32f9a 
Add `threads_per_warp` (equivalent to subgroup size) field to `Config`
so autotuner can use that as an additional parameter during its
workflow.

Signed-off-by: Victor Perez <victor.perez@codeplay.com>
@chengjunlu
Use the autotune with threads_per_warp for softmax.
bd400b0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@chengjunlu @victor-eds