Models 12-2023
Models for the paper "TIMED-Design: Flexible and Accessible Protein Sequence Design with Convolutional Neural Networks"
Performance Comparison
For detailed performance comparisons, please see the paper.
Macro Recall
Macro-Recall is accuracy averaged per residue - resistant to class imbalance.
RMSD_100
We sampled 10% of the PDBench dataset and ran it through AlphaFold2 + Amber relaxation. RMSD_100 is a normalised version of RMSD.
Isoelectric Point Mean Absolute Error (MAE)
Difference between the isoelectric point of the original sequence and the predicted sequence.
Charge Mean Absolute Error (MAE)
Difference between the overall charge of the original sequence and the predicted sequence.
Training
All models were trained using the culled PDB set from PISCES cullpdb_pc90_res3.0_R1.0_d200702_chains40583
containing over 35K non-redundant protein structures (40K+ chains), with resolutions up to 3.0 Å.
CNN Models
We reimplemented all of the CNN models in the literature as they were all closed-source. The dataset for CNN models was created using aposteriori using the following command:
poetry run make-frame-dataset /scratch/datasets/biounit/ -d benchmarking_set.csv -e .pdb1.gz --voxels-per-side 21 --frame-edge-length 21 -g True -p 35 -n benchmark_set -v -r -z -cb True -ae CNOCBCA --compression_gzip True -o /scratch/timed_dataset/
For Charge and Polar models we used the codecs (-ae
) equivalent to CNOCBCAQ
and CNOCBCAP
, respectively.
GNN Models
The code for training ProteinMPNN with custom training sets is not available. We recreated the steps given to us by the authors and published them here: https://github.com/wells-wood-research/ProteinMPNN_custom_training/tree/main