Skip to content

Commit

Permalink
fix: rename geb to dgeb
Browse files Browse the repository at this point in the history
  • Loading branch information
jlkravitz committed Jul 1, 2024
1 parent 4b791ee commit be712f8
Show file tree
Hide file tree
Showing 18 changed files with 31 additions and 33 deletions.
60 changes: 29 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,71 +1,69 @@
<h1 align="center">Genomic Embedding Benchmark</h1>
<h1 align="center">Diverse Genomic Embedding Benchmark</h1>

<p align="center">
<a href="https://github.com/tattabio/geb/releases">
<img alt="GitHub release" src="https://img.shields.io/github/v/release/tattabio/geb.svg">
<a href="https://github.com/tattabio/dgeb/releases">
<img alt="GitHub release" src="https://img.shields.io/github/v/release/tattabio/dgeb.svg">
</a>
<a href="">
<img alt="arXiv URL" src="">
</a>
<a href="https://github.com/tattabio/geb/blob/main/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/tattabio/geb.svg">
<a href="https://github.com/tattabio/dgeb/blob/main/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/tattabio/dgeb.svg">
</a>
<a href="https://pepy.tech/project/geb">
<img alt="Downloads" src="https://static.pepy.tech/personalized-badge/geb?period=total&units=international_system&left_color=grey&right_color=orange&left_text=Downloads">
<a href="https://pepy.tech/project/dgeb">
<img alt="Downloads" src="https://static.pepy.tech/personalized-badge/dgeb?period=total&units=international_system&left_color=grey&right_color=orange&left_text=Downloads">
</a>
</p>

<h4 align="center">
<p>
<a href="#installation">Installation</a> |
<a href="#usage">Usage</a> |
<a href="https://huggingface.co/spaces/GEB">Leaderboard</a> |
<a href="https://huggingface.co/spaces/dgeb">Leaderboard</a> |
<a href="#documentation">Documentation</a> |
<a href="#citing">Citing</a>
<p>
</h4>

<h3 align="center">
<a href="https://huggingface.co/spaces/GEB"><img style="float: middle; padding: 10px 10px 10px 10px;" width="100" height="100" src="./docs/images/tatta_logo.png" /></a>
<a href="https://huggingface.co/spaces/dgeb"><img style="float: middle; padding: 10px 10px 10px 10px;" width="100" height="100" src="./docs/images/tatta_logo.png" /></a>
</h3>

## Installation

TODO(joshua):

```bash
pip install geb
pip install dgeb
```

## Usage

- Using the python script (see [run_geb.py](https://github.com/tattabio/geb/blob/main/run_geb.py)):
- Using the python script (see [run_dgeb.py](https://github.com/tattabio/geb/blob/main/run_geb.py)):

```bash
python run_geb.py --model facebook/esm2_t6_8M_UR50D
python run_dgeb.py --model facebook/esm2_t6_8M_UR50D
```


- Using the python API:

```py
import geb
import dgeb

model = geb.get_model("facebook/esm2_t6_8M_UR50D")
tasks = geb.get_tasks_by_modality(geb.Modality.PROTEIN)
evaluation = geb.GEB(tasks=tasks)
model = dgeb.get_model("facebook/esm2_t6_8M_UR50D")
tasks = dgeb.get_tasks_by_modality(geb.Modality.PROTEIN)
evaluation = dgeb.DGEB(tasks=tasks)
evaluation.run(model, output_folder="results")
```


### Using a custom model

Custom models should be wrapped with the `geb.models.BioSeqTransformer` abstract class, and specify the modality, number of layers, and embedding dimension. See see [models.py](https://github.com/tattabio/geb/blob/main/geb/models.py) for additional examples on custom model loading and inference.

Custom models should be wrapped with the `dgeb.models.BioSeqTransformer` abstract class, and specify the modality, number of layers, and embedding dimension. See see [models.py](https://github.com/tattabio/geb/blob/main/geb/models.py) for additional examples on custom model loading and inference.

```python
import geb
from geb.models import BioSeqTransformer
from geb.modality import Modality
import dgeb
from dgeb.models import BioSeqTransformer
from dgeb.modality import Modality

class MyModel(BioSeqTransformer):

Expand All @@ -83,38 +81,38 @@ class MyModel(BioSeqTransformer):


model = MyModel()
tasks = geb.get_tasks_by_modality(model.modality)
tasks = dgeb.get_tasks_by_modality(model.modality)
evaluation = MTEB(tasks=tasks)
evaluation.run(model)
```

### Evaluating on a custom dataset

TODO(andre): Update this section
TODO(andre): Update this section

To evaluate on a custom task, you can run the following code on your custom task.

```python
import geb
from geb.tasks import AbsTask
import dgeb
from dgeb.tasks import AbsTask

class MyCustomTask(AbsTask):
def run(
self, model: BioSeqTransformer, layers: Optional[List[int]] = None
) -> TaskResult:
pass

model = geb.models.ESM("facebook/esm2_t6_8M_UR50D")
evaluation = geb.GEB(tasks=[MyCustomTask()])
model = dgeb.models.ESM("facebook/esm2_t6_8M_UR50D")
evaluation = dgeb.DGEB(tasks=[MyCustomTask()])
evaluation.run(model)
```

</details>

## Citing

GEB was introduced in "[GEB: Genomic Embedding Benchmark]()", feel free to cite:
dgeb was introduced in "[DGEB: Diverse Genomic Embedding Benchmark]()", feel free to cite:

TODO(andre): bibtex

For works that have used GEB for benchmarking, you can find them on the [leaderboard](https://huggingface.co/spaces/tattabio/GEB/leaderboard).
For works that have used dgeb for benchmarking, you can find them on the [leaderboard](https://huggingface.co/spaces/tattabio/DGEB/leaderboard).
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions plot_benchmarks.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
import pandas as pd
import seaborn as sns

from geb.geb import get_all_tasks, get_output_folder, get_tasks_by_name
from geb.tasks.tasks import TaskResult
from dgeb.geb import get_all_tasks, get_output_folder, get_tasks_by_name
from dgeb.tasks.tasks import TaskResult

ALL_TASKS = [task.metadata.id for task in get_all_tasks()]

Expand Down
File renamed without changes.

0 comments on commit be712f8

Please sign in to comment.