Skip to content

Commit

Permalink
add Puma paper links (#267)
Browse files Browse the repository at this point in the history
  • Loading branch information
vincehong authored Jul 25, 2023
1 parent 8183f81 commit 1926938
Showing 1 changed file with 2 additions and 18 deletions.
20 changes: 2 additions & 18 deletions examples/python/ml/flax_llama7b/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Flax Llama-7B Example with Puma

This example demonstrates how to use SPU to run secure inference on a pre-trained
[Llama-7B](https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/) model.
[Llama-7B](https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/) model using [Puma](https://arxiv.org/abs/2307.12533)

1. Install huggingface transformers library

Expand Down Expand Up @@ -47,20 +47,4 @@ This example demonstrates how to use SPU to run secure inference on a pre-traine
bazel run -c opt //examples/python/ml/flax_llama7b -- --config `pwd`/examples/python/ml/flax_llama7b/3pc.json
```

5. Run `secure inferce of GPT2 with Puma`:

load gpt2 model, replace line 55-57 as follows:

```python
tokenizer = AutoTokenizer.from_pretrained("gpt2")
pretrained_model = FlaxGPT2LMHeadModel.from_pretrained("gpt2")
```

prepare the configuration, replace line 128-129 as follows:

```python
config = GPT2Config()
model = FlaxGPT2LMHeadModel(config=config)
```

The other parts are similar to run `Flax-Llama7B`.
5. To reproduce the benchmarks results in the [Puma paper](https://arxiv.org/abs/2307.12533), please check [here](https://github.com/AntCPLab/puma_benchmarks).

0 comments on commit 1926938

Please sign in to comment.