support Group attention (Llama 2)

[xinhaoc]

Description of changes:

Related Issues:

Linked Issues:

• Issue #

Issues closed by this PR:

• Closes Implement Kernel Fusion Optimizations for IncMHA/SpecMHA/TreeMHA #908

Before merging:

• Did you update the flexflow-third-party repo, if modifying any of the Cmake files, the build configs, or the submodules?

[xinhaoc]

Give three tips for staying healthy.
The first step is to make sure you are getting enough sleep.
The second step is to make sure you are eating healthy foods.
The third step is to make sure you are exercising regularly.
The fourth step is to make sure you are taking care of your mental health.
The fifth step is to make sure you are taking care of your physical health.
The sixth step is to make sure you are taking care of your emotional health.
The seventh step is to make sure you are taking care of your spiritual health.

[xinhaoc]

result from llama 70B:

⁇ Give three tips for staying healthy.

1. Eat a balanced diet. 2. Exercise regularly. 3. Get enough sleep.
2. What is the best way to stay healthy?
3. What are some tips for staying healthy?
4. How can I stay healthy?
5. What are some ways to stay healthy?
6. How can I maintain my health?
7. What are some healthy habits?
8. What are some healthy lifestyle choices?
9. How can I improve my health?

or
Give three tips for staying healthy.
What are the three tips for staying healthy?
What are the 3 tips for staying healthy?
What are the 3 tips for staying healthy Brainly?
What are the 3 tips for staying healthy answer?
What are the 3 tips for staying healthy answer in one word?
What are the 3 tips for staying healthy answer in one word Brainly?
What are the 3 tips for staying healthy answer in one word 2021?
What are the

[jiazhihao]

It seems this PR has passed CI. @xinhaoc @goliaro do you think we can merge it to the inference branch now? In this case, we will add LLaMA 2 support on the Python side later.

[xinhaoc]

One more thing to do, the quantization is not supported yet.

[xinhaoc]

and do you think we should fix this issue #908 in this branch?

[jiazhihao]

We can add the quantization support in a separate PR.

[jiazhihao]

We can add the quantization support in a separate PR.

If the fix is not too hard, I think it's a good idea to fix them in this PR.