QCD transformer demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9c3d576",
   "metadata": {},
   "source": [
    "# SYMBA Model for QCD:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ff828b0",
   "metadata": {},
   "source": [
    "We train this model to map the (amplitude) of particles interactions in quantum chromodynamics theory QCD to the (squared amplitude), which is the key element in calculating the cross section."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677bea40",
   "metadata": {},
   "source": [
    "# "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8f74e1ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras.layers import TextVectorization\n",
    "import re\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6404f2",
   "metadata": {},
   "source": [
    "## Preparing the data:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d25cb46",
   "metadata": {},
   "source": [
    "To prepape the dataset for training, we use the symbolic computation program MARTY [7] to generate expressions for possible interactions in QCD. We restrict the scope 2-to-2 and 2-to-3 particle tree-level processes. All interactions involving off-shell and on-shell particles, anti-particles and gauge bosons are included. Since it is possible for different amplitudes to yield the same squared expressions, we include such amplitudes in our dataset.  All output expressions (squared amplitudes) are simplified with the Python symbolic mathematics module SymPy [25], and factorized by particle masses, and organized into a standard format (first the overall factors, then the terms of the numerator, and third the denominator).\n",
    "\n",
    "You can find the QCD data here:\n",
    "https://drive.google.com/file/d/1s5wRfLnMNo9l8lqn5IWPrDrNGyb_sRfX/view?usp=share_link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ab14b4a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data size:  191857\n"
     ]
    }
   ],
   "source": [
    "with open('qcd_order_data.txt', 'r', encoding='utf-8') as f:\n",
    "    lines = f.read().split('\\n')\n",
    "    \n",
    "\n",
    "\n",
    "text_pairs =[]\n",
    "for line in lines[: min(len(lines), len(lines)-1)]:\n",
    "    intr, amp, sqamp, t  = line.split(':')\n",
    "    sqamp = \"[start] \" + sqamp + \" [end]\"\n",
    "    text_pairs.append((intr, amp,sqamp, float(t) ))\n",
    "    \n",
    "\n",
    "    \n",
    "text_pairs = list(set(text_pairs))\n",
    "random.seed(3333)\n",
    "random.shuffle(text_pairs)\n",
    "\n",
    "\n",
    "print('data size: ', len(text_pairs))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce5a434c",
   "metadata": {},
   "source": [
    "#### Examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "550bec71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Amplitude:   \n",
      "   1/12*i*g^3*(m_u*gamma_{+%\\sigma,%eps,%del}*gamma_{%\\sigma,%eta,%del}*gamma_{%\\lambda,%eta,%eta}*(delta_{%E,%H}*T_C_10_{%f,%F,%G} + (-3)*delta_{%E,%G}*T_C_10_{%f,%F,%H})*G_{j,%f,+%\\lambda}(p_2)*c_{i,%E,%eps}(p_1)^(*)*c_{l,%G,%del}(p_4)*u_{k,%F,%eta}(p_3)^(*)*u_{i,%H,%del}(p_5) + -p_2_%\\lambda*gamma_{+%\\sigma,%eps,%del}*gamma_{%\\sigma,%eta,%del}*gamma_{%\\lambda,%eta,%eta}*gamma_{+%\\lambda,%eta,%eta}*(delta_{%E,%H}*T_C_10_{%f,%F,%G} + (-3)*delta_{%E,%G}*T_C_10_{%f,%F,%H})*G_{j,%f,+%\\lambda}(p_2)*c_{i,%E,%eps}(p_1)^(*)*c_{l,%G,%del}(p_4)*u_{k,%F,%eta}(p_3)^(*)*u_{i,%H,%del}(p_5) + p_3_%\\lambda*gamma_{+%\\sigma,%eps,%del}*gamma_{%\\sigma,%eta,%del}*gamma_{%\\lambda,%eta,%eta}*gamma_{+%\\lambda,%eta,%eta}*(delta_{%E,%H}*T_C_10_{%f,%F,%G} + (-3)*delta_{%E,%G}*T_C_10_{%f,%F,%H})*G_{j,%f,+%\\lambda}(p_2)*c_{i,%E,%eps}(p_1)^(*)*c_{l,%G,%del}(p_4)*u_{k,%F,%eta}(p_3)^(*)*u_{i,%H,%del}(p_5))/((m_c^2 + -s_14)*(m_u^2 + 2*s_23 + -s_33))  \n",
      "\n",
      "Squared Amplitude:   \n",
      " [start] ([m_u^4, m_u^2, 1], [8*(2*m_c^2 - s_14), -4*(m_c^2*(4*s_23 + 4*s_25 - 3*s_35) - 4*s_12*s_45 + 3*s_13*s_45 - 2*s_14*s_23 - 4*s_15*s_24 + 3*s_15*s_34), -4*(m_c^2*(2*s_23*s_25 - 2*s_25*s_33 + s_33*s_35) - 2*s_12*s_23*s_45 + 2*s_12*s_33*s_45 - s_13*s_33*s_45 - 2*s_15*s_23*s_24 + 2*s_15*s_24*s_33 - s_15*s_33*s_34)], [9*(m_c^2 - s_14)^2*(m_u^2 + 2*s_23 - s_33)^2]) [end]\n"
     ]
    }
   ],
   "source": [
    "print('Amplitude:  ' , '\\n', text_pairs[1][1], '\\n' )\n",
    "print('Squared Amplitude:  ', '\\n' ,text_pairs[1][2] )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a38e6678",
   "metadata": {},
   "source": [
    "Remove long amplitude/square amplitude:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dff207cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data size:  140094\n"
     ]
    }
   ],
   "source": [
    "text_pairs1 = []\n",
    "\n",
    "for i in range(len(text_pairs)):\n",
    "    if len(text_pairs[i][1]) < 2000  and len(text_pairs[i][2]) < 1800:\n",
    "        text_pairs1.append(text_pairs[i])\n",
    "\n",
    "text_pairs = text_pairs1\n",
    "print('data size: ', len(text_pairs))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a53c810c",
   "metadata": {},
   "source": [
    "### Data preprocessing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6413a9af",
   "metadata": {},
   "outputs": [],
   "source": [
    "#preprocessing for the amplitudes:\n",
    "\n",
    "def prepro_ampl(data):\n",
    "\n",
    "    for r in (('}', '}'),('{', ' {'), (' + ',' +' ), (' - ', ' -') ,('*', '* '), ('(* )', '(*)'),('^', '^') , ('(', ' ('),(')', ')'),('/', ' /')  ,('  ', ' ') ) :  #,('{', ' {')\n",
    "        data = data.replace(*r) \n",
    "        \n",
    "    return data\n",
    "\n",
    "#preprocessing for the squared amplitudes:\n",
    "\n",
    "def prepro_squared_ampl(data):\n",
    "\n",
    "    for r in (('*', '*'), (',', ' , '), ('*(', ' *( ') , ('([', '[ '), ('])', ' ]'), ('[', '[ '), (']', ' ]'), ('[ start ]', '[start]'), ('[ end ]', '[end]'), (' - ', ' -'), (' + ',' +' ) ,('/', ' / ') ,('  ', ' ')) :\n",
    "        data = data.replace(*r) \n",
    "    data = re.sub(r\"\\*(s_\\d+\\*s_\\d+)\", r\"* \\1\", data)\n",
    "    data = re.sub(r\"\\*(s_\\d+\\^\\d+\\*s_\\d+)\", r\"* \\1\", data)\n",
    "    data = re.sub(r\"\\*(m_\\w+\\^\\d+\\*s_\\d+)\", r\"* \\1\", data)\n",
    "    data = re.sub(r\"(m_\\w+\\^\\d+)\", r\" \\1 \", data)\n",
    "    data = data.replace('  ', ' ')\n",
    "    \n",
    "    \n",
    "    \n",
    "    return data\n",
    "\n",
    "\n",
    "\n",
    "text_pairs_prep = []\n",
    "for i in range(len(text_pairs)):\n",
    "    text_pairs_prep.append((text_pairs[i][0], prepro_ampl(text_pairs[i][1]), prepro_squared_ampl(text_pairs[i][2]), text_pairs[i][3]))\n",
    "\n",
    "text_pairs = text_pairs_prep"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2d8f3e5",
   "metadata": {},
   "source": [
    "###  Maximum sequence length:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4f38dfea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maximum sequence length of amplitudes        : 251\n",
      "Maximum sequence length of squared amplitudes: 249\n"
     ]
    }
   ],
   "source": [
    "def max_len(sq_data):\n",
    "    l = len(sq_data[sq_data.index(max(sq_data, key=len))].split())\n",
    "    return l\n",
    "\n",
    "\n",
    "ampl = [pair[1] for pair in text_pairs]\n",
    "sq_ampl= [pair[2] for pair in text_pairs]\n",
    "\n",
    "print( 'Maximum sequence length of amplitudes        :' ,max_len(ampl))\n",
    "print( 'Maximum sequence length of squared amplitudes:' ,max_len(sq_ampl))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b42fb7da",
   "metadata": {},
   "source": [
    "\n",
    "# Tokenization:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bf843f6",
   "metadata": {},
   "source": [
    "### Split the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ffe743a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_val_samples = int(0.15 * len(text_pairs))\n",
    "num_train_samples = len(text_pairs) - 2 * num_val_samples\n",
    "train_pairs = text_pairs[:num_train_samples]\n",
    "val_pairs  = text_pairs[num_train_samples : num_train_samples + num_val_samples]\n",
    "test_pairs = text_pairs[num_train_samples + num_val_samples :]\n",
    "\n",
    "train_input_texts = [pair[1] for pair in train_pairs]\n",
    "train_output_texts = [pair[2] for pair in train_pairs]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f1a8ccd",
   "metadata": {},
   "source": [
    "### Tokeniaztion:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c53d1191",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-12-07 11:24:22.408490: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of input (amplitude) tokens:   2872\n",
      "number of target (squared amplitude) tokens:  1988\n"
     ]
    }
   ],
   "source": [
    "vocab_size = 2875\n",
    "sequence_length = 256\n",
    "batch_size = 64\n",
    "\n",
    "input_vectorization = TextVectorization(\n",
    "    max_tokens=vocab_size, output_mode=\"int\", output_sequence_length=sequence_length, standardize=None, )\n",
    "\n",
    "output_vectorization = TextVectorization(\n",
    "    max_tokens=vocab_size,\n",
    "    output_mode=\"int\",\n",
    "    output_sequence_length=sequence_length + 1, standardize=None)\n",
    "\n",
    "input_vectorization.adapt(train_input_texts)\n",
    "output_vectorization.adapt(train_output_texts)\n",
    "\n",
    "target_tokens = output_vectorization.get_vocabulary()\n",
    "input_tokens = input_vectorization.get_vocabulary()\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "print('number of input (amplitude) tokens:  ', len(input_tokens))\n",
    "print('number of target (squared amplitude) tokens: ', len(target_tokens))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6ed8d80d",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def format_dataset(input_exp, target_exp):\n",
    "    input_exp = input_vectorization(input_exp)\n",
    "    target_exp = output_vectorization(target_exp)\n",
    "    return ({\"encoder_inputs\": input_exp, \"decoder_inputs\": target_exp[:, :-1],}, target_exp[:, 1:])\n",
    "\n",
    "\n",
    "def make_dataset(pairs):\n",
    "    intr, ampl_texts, sqampl_texts, t = zip(*pairs)\n",
    "    ampl_texts = list(ampl_texts)\n",
    "    sqampl_texts = list(sqampl_texts)\n",
    "    dataset = tf.data.Dataset.from_tensor_slices((ampl_texts, sqampl_texts))\n",
    "    dataset = dataset.batch(batch_size)\n",
    "    dataset = dataset.map(format_dataset)\n",
    "    return dataset.shuffle(2048).prefetch(16).cache()\n",
    "\n",
    "\n",
    "train_ds = make_dataset(train_pairs)\n",
    "val_ds = make_dataset(val_pairs)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f439937",
   "metadata": {},
   "source": [
    "## Transformer Model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "410fbb01",
   "metadata": {},
   "outputs": [],
   "source": [
    "class TransformerEncoder(layers.Layer):\n",
    "    def __init__(self, embed_dim, dense_dim, num_heads, **kwargs):\n",
    "        super(TransformerEncoder, self).__init__(**kwargs)\n",
    "        self.embed_dim = embed_dim\n",
    "        self.dense_dim = dense_dim\n",
    "        self.num_heads = num_heads\n",
    "        self.attention = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.dense_proj = keras.Sequential(\n",
    "            [layers.Dense(dense_dim, activation=\"relu\"), layers.Dense(embed_dim),]\n",
    "        )\n",
    "        self.layernorm_1 = layers.LayerNormalization()\n",
    "        self.layernorm_2 = layers.LayerNormalization()\n",
    "        self.supports_masking = True\n",
    "\n",
    "    def call(self, inputs, mask=None):\n",
    "        if mask is not None:\n",
    "            padding_mask = tf.cast(mask[:, tf.newaxis, tf.newaxis, :], dtype=\"int64\")\n",
    "        attention_output = self.attention(\n",
    "            query=inputs, value=inputs, key=inputs, attention_mask=padding_mask\n",
    "        )\n",
    "        proj_input = self.layernorm_1(inputs + attention_output)\n",
    "        proj_output = self.dense_proj(proj_input)\n",
    "        return self.layernorm_2(proj_input + proj_output)\n",
    "\n",
    "\n",
    "class PositionalEmbedding(layers.Layer):\n",
    "    def __init__(self, sequence_length, vocab_size, embed_dim, **kwargs):\n",
    "        super(PositionalEmbedding, self).__init__(**kwargs)\n",
    "        self.token_embeddings = layers.Embedding(\n",
    "            input_dim=vocab_size, output_dim=embed_dim\n",
    "        )\n",
    "        self.position_embeddings = layers.Embedding(\n",
    "            input_dim=sequence_length, output_dim=embed_dim\n",
    "        )\n",
    "        self.sequence_length = sequence_length\n",
    "        self.vocab_size = vocab_size\n",
    "        self.embed_dim = embed_dim\n",
    "\n",
    "    def call(self, inputs):\n",
    "        length = tf.shape(inputs)[-1]\n",
    "        positions = tf.range(start=0, limit=length, delta=1)\n",
    "        embedded_tokens = self.token_embeddings(inputs)\n",
    "        embedded_positions = self.position_embeddings(positions)\n",
    "        return embedded_tokens + embedded_positions\n",
    "\n",
    "    def compute_mask(self, inputs, mask=None):\n",
    "        return tf.math.not_equal(inputs, 0)\n",
    "\n",
    "\n",
    "class TransformerDecoder(layers.Layer):\n",
    "    def __init__(self, embed_dim, latent_dim, num_heads, **kwargs):\n",
    "        super(TransformerDecoder, self).__init__(**kwargs)\n",
    "        self.embed_dim = embed_dim\n",
    "        self.latent_dim = latent_dim\n",
    "        self.num_heads = num_heads\n",
    "        self.attention_1 = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.attention_2 = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.dense_proj = keras.Sequential(\n",
    "            [layers.Dense(latent_dim, activation=\"relu\"), layers.Dense(embed_dim),]\n",
    "        )\n",
    "        self.layernorm_1 = layers.LayerNormalization()\n",
    "        self.layernorm_2 = layers.LayerNormalization()\n",
    "        self.layernorm_3 = layers.LayerNormalization()\n",
    "        self.supports_masking = True\n",
    "\n",
    "    def call(self, inputs, encoder_outputs, mask=None):\n",
    "        causal_mask = self.get_causal_attention_mask(inputs)\n",
    "        if mask is not None:\n",
    "            padding_mask = tf.cast(mask[:, tf.newaxis, :], dtype=\"int64\")\n",
    "            padding_mask = tf.minimum(padding_mask, causal_mask)\n",
    "\n",
    "        attention_output_1 = self.attention_1(\n",
    "            query=inputs, value=inputs, key=inputs, attention_mask=causal_mask\n",
    "        )\n",
    "        out_1 = self.layernorm_1(inputs + attention_output_1)\n",
    "\n",
    "        attention_output_2 = self.attention_2(\n",
    "            query=out_1,\n",
    "            value=encoder_outputs,\n",
    "            key=encoder_outputs,\n",
    "            attention_mask=padding_mask,\n",
    "        )\n",
    "        out_2 = self.layernorm_2(out_1 + attention_output_2)\n",
    "\n",
    "        proj_output = self.dense_proj(out_2)\n",
    "        return self.layernorm_3(out_2 + proj_output)\n",
    "\n",
    "    def get_causal_attention_mask(self, inputs):\n",
    "        input_shape = tf.shape(inputs)\n",
    "        batch_size, sequence_length = input_shape[0], input_shape[1]\n",
    "        i = tf.range(sequence_length)[:, tf.newaxis]\n",
    "        j = tf.range(sequence_length)\n",
    "        mask = tf.cast(i >= j, dtype=\"int64\")\n",
    "        mask = tf.reshape(mask, (1, input_shape[1], input_shape[1]))\n",
    "        mult = tf.concat(\n",
    "            [tf.expand_dims(batch_size, -1), tf.constant([1, 1], dtype=tf.int32)],\n",
    "            axis=0,\n",
    "        )\n",
    "        return tf.tile(mask, mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96d1c4b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "embed_dim = 512\n",
    "latent_dim = 16384\n",
    "num_heads = 8\n",
    "\n",
    "encoder_inputs = keras.Input(shape=(None,), dtype=\"int64\", name=\"encoder_inputs\")\n",
    "x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(encoder_inputs)\n",
    "encoder_outputs = TransformerEncoder(embed_dim, latent_dim, num_heads)(x)\n",
    "encoder = keras.Model(encoder_inputs, encoder_outputs)\n",
    "\n",
    "decoder_inputs = keras.Input(shape=(None,), dtype=\"int64\", name=\"decoder_inputs\")\n",
    "encoded_seq_inputs = keras.Input(shape=(None, embed_dim), name=\"decoder_state_inputs\")\n",
    "x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(decoder_inputs)\n",
    "x = TransformerDecoder(embed_dim, latent_dim, num_heads)(x, encoded_seq_inputs)\n",
    "x = layers.Dropout(0.5)(x)\n",
    "decoder_outputs = layers.Dense(vocab_size, activation=\"softmax\")(x)\n",
    "decoder = keras.Model([decoder_inputs, encoded_seq_inputs], decoder_outputs)\n",
    "\n",
    "decoder_outputs = decoder([decoder_inputs, encoder_outputs])\n",
    "transformer = keras.Model(\n",
    "    [encoder_inputs, decoder_inputs], decoder_outputs, name=\"transformer\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94c9b0a2",
   "metadata": {},
   "source": [
    "## Training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1e738b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 5   #at least 30 epochs\n",
    "learning_rate=0.0001\n",
    "\n",
    "opt = keras.optimizers.Adam(learning_rate=learning_rate)\n",
    "\n",
    "transformer.summary()\n",
    "transformer.compile(loss=\"sparse_categorical_crossentropy\", metrics=[\"accuracy\"], optimizer=opt)\n",
    "transformer.fit(train_ds, epochs=epochs, validation_data=val_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3375bed",
   "metadata": {},
   "source": [
    "## Inference:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65ebca3b",
   "metadata": {},
   "source": [
    "Greedy decoding:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42503048",
   "metadata": {},
   "outputs": [],
   "source": [
    "target_tokens = output_vectorization.get_vocabulary()\n",
    "target_index_lookup = dict(zip(range(len(target_tokens)), target_tokens))\n",
    "max_decoded_sentence_length = max_sequence_length\n",
    "\n",
    "\n",
    "def decode_sequence(input_sentence):\n",
    "    tokenized_input_sentence = input_vectorization([input_sentence])\n",
    "    decoded_sentence = \"[start]\"\n",
    "    for i in range(max_decoded_sentence_length):\n",
    "        tokenized_target_sentence = output_vectorization([decoded_sentence])[:, :-1]\n",
    "        predictions = transformer([tokenized_input_sentence, tokenized_target_sentence])\n",
    "\n",
    "        sampled_token_index = np.argmax(predictions[0, i, :])\n",
    "        sampled_token = target_index_lookup[sampled_token_index]\n",
    "        decoded_sentence += \" \" + sampled_token\n",
    "\n",
    "        if sampled_token == \"[end]\":\n",
    "            break\n",
    "    return decoded_sentence\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6952438b",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_input_texts = [pair[1] for pair in test_pairs]\n",
    "test_output_texts = [pair[2] for pair in test_pairs]\n",
    "marty_time = [pair[3] for pair in test_pairs]\n",
    "\n",
    "\n",
    "\n",
    "for i in random.sample(range(0,len(test_input_texts)), 5):\n",
    "    input_sentence = test_input_texts[i]\n",
    "    start = time.process_time()\n",
    "    translated = decode_sequence(input_sentence)\n",
    "    elapsed = (time.process_time() - start)\n",
    "    print('Actual:    ', test_output_texts[i], '\\n')\n",
    "    print('Predicted: ', translated, '\\n')\n",
    "    print(test_output_texts[i]==translated, '\\n')\n",
    "    print('symba time: ', elapsed, '\\n')\n",
    "    print('Marty time: ', str(marty_time[i]), '\\n')\n",
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}