Siamese Network Training and Evaluation

Training process

To train the siamese network you have to use the ./train_siamese.py script. There are a couple of important settings to setup:

• base model selection ./train_siamese.py:L13 select one of three available bases [MobileNetV2, ResNet101V2, EfficientNetB5]

base_model = list(base_models.keys())[0]  # MobileNetV2

• network prefix ./train_siamese.py:L20 is an additional identification to add to the weights name

    prefix = "block3c"

• base_layer_name - can be passed as a parameter in ./train_siamese.py:L19, has to be a layer from the selected base network, it is used as feature extraction layer. Each base model has default layer so if you just want to use default one don't setup this parameter.
• layers_to_train - a list of layer prefixes to enable training. Can be passed as a parameter in ./train_siamese.py:L19. By default base layers are not trained, only top layers are. If you want to train them just paste list of prefixes like ["conv2", "conv3"] and all layers with name starting with those prefixes are going to be retrained. Each base model has default layer so if you just want to use default one don't setup this parameter.
• learning_rate - you know what this is and what it does, you can change its value in model/siamese/config.py:L26 (some base models just work better with different LR)
• batch_size - you can change its value in model/siamese/config.py:L25, min value is 16 and you should set it with multiples of 16s (32, 64). It is important because Triplet Loss has to be able to distinguish classes
• epochs - model/siamese/config.py:L24, i dont' have to explain this.

During training best weight are going to be saved in model/siamese/weights/BASE_MODEL_NAME/WEIGHT_NAME.h5. You need them for evaluation.

Embedding space generation

Evaluation is done by executing:

python generate_siamese_emb_space.py

To generate embedding vectors use generate_siamese_emb_space.py, as in the previous example you have to change a couple of things.

• base model selection ./generate_siamese_emb_space.py:L24 select one of three available bases [MobileNetV2, ResNet101V2, EfficientNetB5]

base_model = list(base_models.keys())[0]  # MobileNetV2

• base_layer_name - can be passed as a parameter in ./generate_siamese_emb_space.py:L30, has to be a layer from the selected base network, it is used as feature extraction layer. Each base model has default layer so if you just want to use default one don't setup this parameter.
• saved_weights_file - ./generate_siamese_emb_space.py:L14, this is only a filename, don't worry about the path because it's already in model/siamese/weights/BASE_MODEL_NAME/

This is going to produce four files:

• vecs-train-BASE_MODEL_NAME.tsv - list of embeddings for test dataset
• meta-train-BASE_MODEL_NAME.tsv - list of labels for embeddings
• vecs-conc-BASE_MODEL_NAME.tsv - avg of all embeddings
• meta-conc-BASE_MODEL_NAME.tsv - classes for avg embeddings

You have to copy last two into model/siamese/vectors folder.

Siamese Evaluation

Now you can run an evaluation of the siamese network. As in the previous example, there is a basic setup of your network:

• base model selection ./test_siamese.py:L43 select one of three available bases [MobileNetV2, ResNet101V2, EfficientNetB5]

base_model = list(base_models.keys())[0]  # MobileNetV2

• path/to/vecs-conc-BASE_MODEL_NAME.tsv ./test_siamese.py:L31 path to previously copied avg vectors file

flags.DEFINE_string(
    "vectors",
    "model/siamese/vectors/vecs-conc-BASE_MODEL_NAME.tsv",
    "path to vectors tsv",
)

• path/to/meta-conc-BASE_MODEL_NAME.tsv ./test_siamese.py:L37 path to previously copied meta

flags.DEFINE_string(
    "meta",
    "model/siamese/vectors/meta-conc-BASE_MODEL_NAME.tsv",
    "path to meta tsv",
)

• base_layer_name - can be passed as a parameter in ./test_siamese.py:L69, has to be a layer from the selected base network, it is used as feature extraction layer. Each base model has default layer so if you just want to use default one don't setup this parameter.
• saved_weights_file - ./test_siamese.py:L19, this is only a filename, don't worry about the path because it's already in model/siamese/weights/BASE_MODEL_NAME/

After setup is done just run

python test_siamese.py

And your evaluation results are going to be stored in ./experiments/siamese/BASE_MODEL_NAME/DATE/ folder, each run generates conf matrix and class report.