extra fine-tuning on dataset of dog/cat antibodies

[avilella]

Hi,

We have a set of dog and cat antibodies (about 200,000) which we would like to use to do some extra fine-tuning to the model in this software, and we are wondering about the feasibility of doing this, considering the Nature Biotech paper describes evolving human antibodies, rather than dog or cat antibodies.

What would be a good starting point and computational requirements (Nvidia GPUs) for this mini-project? Thanks in advance.

[brianhie]

Hi @avilella, it is hard to say how many computational resources are required to finetune the ESM-1 models -- this will vary depending on your overall task and performance desired. Here are some statistics on how much compute was required to pretrain the ESM-1b models: