[WIP] Pipeline Parallelism implementation. #502

Job	Run time
CodeQL Scan	3m 1s
Dependency Review	5s
Calculate Tests To Trigger	4s
Test / gsm8k_small_g3_tp1	49m 24s
Test / gsm8k_small_g3_tp2	6m 1s
Test / gsm8k_small_g2_tp1	1m 3s
Test / gsm8k_small_g2_tp2	1m 3s
Test / gsm8k_large_g3_tp2	7m 10s
Test / gsm8k_large_g2_tp4	1m 3s
Test / gsm8k_small_g3_tp1_fp8	6m 19s
Test / gsm8k_small_g3_tp2_fp8	3m 57s
Test / gsm8k_small_g3_tp1_mss	4m 46s
Test / gsm8k_small_g2_tp1_mss	6m 59s
Test / gsm8k_small_g3_tp2_mss	3m 8s
Test / gsm8k_small_g2_tp2_mss	5m 17s
Test / gsm8k_small_g2_tp1_spec_decode	6m 7s
Test / gsm8k_small_g2_tp1_mlp_spec_decode	9m 14s
Test / gsm8k_small_g2_tp1_medusa_spec_decode	8m 0s
Test / gsm8k_small_g2_tp1_eagle_spec_decode	4m 51s
Test / test_llama_lora	1m 3s
Test / test_multilora	1m 3s
Test / test_long_context	1m 2s
	2h 10m 40s

Provide feedback