latent_dirichlet_allocation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "<h1>LDA (Latent Dirichlet Allocation) Manual</h1>\n",
        "<br>\n",
        "<p>\n",
        "Pada bagian ini penulis akan menjelaskan pembuatan model LDA dengan metode manual dan implementasinya menggunakan code python. Sistematis perhitungan dari model Latent Dirichlet Allocation yang penulis dapatkan dari (Harsh Bansal, 2020) secara manual dapat dirincikan dengan beberapa tahapan sebagai berikut :\n",
        "</p>\n",
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Contoh jika kita mengambil salah satu kalimat dari dataset yang dimiliki <i><b>“cant login even though change password”</b></i><h4></li>\n",
        "</ul>\n",
        "<br>"
      ],
      "metadata": {
        "id": "zCUwkLquzy3_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "topics = [\n",
        "    \"cant login even though change password\"\n",
        "    # .....\n",
        "    # .....\n",
        "    # ..... Anggap aja ada 300 juta data, aowkwkkw...\n",
        "]\n",
        "\n",
        "tokenize = [\n",
        "    re.findall(r'\\b\\w+\\b', topic.lower()) for topic in topics\n",
        "]\n",
        "\n",
        "print(f\"tokenize: {tokenize}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eJtakw4bBzjo",
        "outputId": "77f16959-5a68-42d3-be5e-909f2d49880d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tokenize: [['cant', 'login', 'even', 'though', 'change', 'password']]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Tahapan selanjutnya adalah menentukan jumlah topik yang diinginkan, misal dalam penelitian ini penulis menggunakan jumlah topik tiga (3).</h4></li>\n",
        "</ul>\n",
        "<br>"
      ],
      "metadata": {
        "id": "epB_kZ-GBuuL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "topic_total = 3"
      ],
      "metadata": {
        "id": "j6qSpT9EQYvU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<h3>Libraries</h3>"
      ],
      "metadata": {
        "id": "MRASfy4V6lwf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "from IPython.display import HTML"
      ],
      "metadata": {
        "id": "N_tJOC0f4fYH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Jika sudah menentukan jumlah topik yang diinginkan, selanjutnya model LDA akan secara acak menyebarkan kata-kata yang dimiliki kalimat tersebut ke masing-masing <i>num_topic</i> misal dalam hal ini menggunakan dua dokumen.</h4></li>\n",
        "</ul>\n",
        "<br>\n",
        "<p><i>Tabel 4.9 Penyebaran Acak Kata Pada Dokumen</i></p>\n",
        "<br>"
      ],
      "metadata": {
        "id": "dPzhXNO46Y8b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "document_topic_matrix = [\n",
        "    [2, 3, 1],\n",
        "    [2, 2, 2]\n",
        "]\n",
        "\n",
        "def show_table_document_topic_matrix():\n",
        "  topic_matrix = []\n",
        "\n",
        "  for k_topic, v_topic in enumerate(document_topic_matrix):\n",
        "    topic_matrix.append(\n",
        "        [f\"Doc {k_topic + 1}\" if x == 0 else v_topic[x - 1] for x in range(len(v_topic) + 1)]\n",
        "    )\n",
        "\n",
        "  return pd.DataFrame(\n",
        "    topic_matrix,\n",
        "    columns = ['Doc', 'T1', 'T2', 'T3']\n",
        "  ).to_html(index=False)\n",
        "\n",
        "\n",
        "\n",
        "HTML(show_table_document_topic_matrix())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "OZ0iF0u-0Mgd",
        "outputId": "f378a0eb-61f9-4b68-a888-58aa9a6d059e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>Doc</th>\n",
              "      <th>T1</th>\n",
              "      <th>T2</th>\n",
              "      <th>T3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>Doc 1</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>Doc 2</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Tahapan selanjutnya adalah menghitung semua kata unik tersebut yang berasal dari semua dokumen pada topik yang ditentukan</h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "p9WA1jiJA414"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "words_topic_matrix = [\n",
        "    [8, 2, 20],\n",
        "    [13, 10, 11],\n",
        "    [11, 14, 23],\n",
        "    [15, 24, 4],\n",
        "    [10, 15, 11],\n",
        "    [7, 11, 5]\n",
        "]\n",
        "\n",
        "def show_table_words_topic_matrix():\n",
        "  topic_matrix = []\n",
        "\n",
        "  for k_topic, v_topic in enumerate(words_topic_matrix):\n",
        "      topic_matrix.append(\n",
        "          [f\"{tokenize[0][k_topic]}\" if x == 0 else v_topic[x - 1] for x in range(len(v_topic) + 1)]\n",
        "      )\n",
        "\n",
        "  return pd.DataFrame(\n",
        "    topic_matrix,\n",
        "    columns = ['Doc', 'T1', 'T2', 'T3']\n",
        "  ).to_html(index=False)\n",
        "\n",
        "\n",
        "\n",
        "HTML(show_table_words_topic_matrix())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "id": "5HjzgKaNBBCX",
        "outputId": "6ceef562-238f-41fc-f736-9eb1fc38386e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>Doc</th>\n",
              "      <th>T1</th>\n",
              "      <th>T2</th>\n",
              "      <th>T3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>cant</td>\n",
              "      <td>8</td>\n",
              "      <td>2</td>\n",
              "      <td>20</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>login</td>\n",
              "      <td>13</td>\n",
              "      <td>10</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>even</td>\n",
              "      <td>11</td>\n",
              "      <td>14</td>\n",
              "      <td>23</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>though</td>\n",
              "      <td>15</td>\n",
              "      <td>24</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>change</td>\n",
              "      <td>10</td>\n",
              "      <td>15</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>password</td>\n",
              "      <td>7</td>\n",
              "      <td>11</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Pengurangan hitungan sebesar 1 dari <i>document topic matrix</i></h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "aZMacov9PdoD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for k_document_topic, v_document_topic in enumerate(document_topic_matrix):\n",
        "    for k_topic, v_topic in enumerate(v_document_topic):\n",
        "        document_topic_matrix[k_document_topic][k_topic] -= 1\n",
        "\n",
        "HTML(show_table_document_topic_matrix())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "3P9Y8nKpP9w5",
        "outputId": "392a3972-9290-4d22-fbd3-8e8fdf8ed029"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>Doc</th>\n",
              "      <th>T1</th>\n",
              "      <th>T2</th>\n",
              "      <th>T3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>Doc 1</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>Doc 2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Pengurangan hitungan sebesar 1 dari <i>words topic matrix</i></h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "8SU11uQdSpvI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for k_word_topic, v_word_topic in enumerate(words_topic_matrix):\n",
        "    for k_topic, v_topic in enumerate(v_word_topic):\n",
        "        words_topic_matrix[k_word_topic][k_topic] -= 1\n",
        "\n",
        "HTML(show_table_words_topic_matrix())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "id": "DukSMZZ7S4Bi",
        "outputId": "24b3884d-873a-4e30-c19e-63bfef84ceea"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>Doc</th>\n",
              "      <th>T1</th>\n",
              "      <th>T2</th>\n",
              "      <th>T3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>cant</td>\n",
              "      <td>7</td>\n",
              "      <td>1</td>\n",
              "      <td>19</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>login</td>\n",
              "      <td>12</td>\n",
              "      <td>9</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>even</td>\n",
              "      <td>10</td>\n",
              "      <td>13</td>\n",
              "      <td>22</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>though</td>\n",
              "      <td>14</td>\n",
              "      <td>23</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>change</td>\n",
              "      <td>9</td>\n",
              "      <td>14</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>password</td>\n",
              "      <td>6</td>\n",
              "      <td>10</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Menghitung $P(Topic\\space T \\space|\\space Document \\space D )$ atau $P(t_k\\space| D_i)$</h4></li>\n",
        "</ul>\n",
        "<br>\n",
        "<p>Perhitungan ini untuk menghitung proporsi kata dalam dokumen $D$ yang saat ini ditugaskan ke topik $T$. Rumus dari perhitungan ini adalah sebagai berikut :</p>\n",
        "<br>\n",
        "<p>$P(t_k\\space| D_i) = \\frac{n_{i,k} \\space+\\space {\\alpha}}{N_i\\space-\\space1\\space+\\space k_{\\alpha}}$</p>\n",
        "<p>Dimana :</p>\n",
        "<ul>\n",
        "  <li>$n_{i,k}$ = total kata dalam i doc dan k topic</li>\n",
        "  <li>$\\alpha$ = <i>hyper parameter</i>, pada contoh ini menggunakan nilai $0.1$</li>\n",
        "  <li>$N_i$ = total jumlah kata pada i document</li>\n",
        "  <li>$K$ = total topik yang ditentukan</li>\n",
        "</ul>\n",
        "<br>\n",
        "<p>perhitungannya adalah sebagai berikut :</p>"
      ],
      "metadata": {
        "id": "0EdM5V4kaRrn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def calculate_topic_document_probability(doc_idx, topic_idx, doc_length):\n",
        "    ALPHA = 0.1\n",
        "    # (n_ik + a) / ((N_i - 1) + K * a)\n",
        "    return (document_topic_matrix[doc_idx][topic_idx] + ALPHA) / (doc_length - 1 + topic_total * ALPHA)"
      ],
      "metadata": {
        "id": "DdUdssElf-Q7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h4><i><b>Simulasi :</b></i></h4>"
      ],
      "metadata": {
        "id": "Lrkx-zhc0pCL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# memilih urutan Doc ke 1 == idx ke 0\n",
        "doc = 0\n",
        "\n",
        "# memilih urutan topik ke-1 == idx ke 0\n",
        "topic = 0\n",
        "\n",
        "\n",
        "for k_document, v_document in enumerate(tokenize):\n",
        "  probability = calculate_topic_document_probability(\n",
        "      doc_idx = doc,\n",
        "      topic_idx = topic,\n",
        "      doc_length = len(v_document),\n",
        "  )\n",
        "\n",
        "  print(\n",
        "      f\"[ Doc {k_document + 1} ]\\n\" +\n",
        "      f\"Data: {v_document}\\n\" +\n",
        "      f\"n_ik: {document_topic_matrix[doc][topic]}\\n\" +\n",
        "      f\"N_i: {len(v_document)}\\n\" +\n",
        "      f\"Total Topic: {topic_total}\\n\" +\n",
        "      f\"{'-'*10}\\nProbablility: {probability}\\n\\n\"\n",
        "  )\n",
        "\n",
        "  calculate_topic_document_probability_result = probability\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UjSH8OZH028d",
        "outputId": "07afeb36-34c6-4af6-911e-ba64d446014f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[ Doc 1 ]\n",
            "Data: ['cant', 'login', 'even', 'though', 'change', 'password']\n",
            "n_ik: 1\n",
            "N_i: 6\n",
            "Total Topic: 3\n",
            "----------\n",
            "Probablility: 0.2075471698113208\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Menghitung $P(Word\\space W \\space|\\space Topic \\space T )$ atau $P(w_j\\space| T_k)$</h4></li>\n",
        "</ul>\n",
        "<br>\n",
        "<p>Perhitungan ini untuk menghitung proporsi penugasan ke topik $T$ atas semua dokumen yang berasal dari kata $W$. Rumus dari perhitungan ini adalah sebagai berikut :</p>\n",
        "<br>\n",
        "<p>$P(w_j\\space| T_k) = \\frac{m_{j,k} \\space+\\space {\\beta}}\n",
        "{\\Sigma_{j=v}m_{j,k}\\space+\\space V\\beta}$</p>\n",
        "<p>Dimana :</p>\n",
        "<ul>\n",
        "  <li>$m_{j,k}$ = berapa kali kata $W$ muncul pada topik $T$</li>\n",
        "  <li>$\\beta$ = <i>hyper parameter</i>, pada contoh ini menggunakan nilai $0.1$</li>\n",
        "  <li>$V$ = total kata yang terdapat pada topik $T$</li>\n",
        "</ul>\n",
        "<br>\n",
        "<p>perhitungannya adalah sebagai berikut :</p>"
      ],
      "metadata": {
        "id": "iSM8XG9msY3k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Login (word: 2 -> idx: 1; topic: 1 -> idx: 0)\n",
        "print(f\"Login Word: {words_topic_matrix[1][0]}\")\n",
        "\n",
        "# Perhitungan Sigma words (topic: 1 -> idx: 0)\n",
        "total = 0\n",
        "for word_topic_matrix in words_topic_matrix :\n",
        "  total += word_topic_matrix[0]\n",
        "\n",
        "print(f\"peritungan sigma: {total}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e25YE16p0Sbb",
        "outputId": "1d5af32b-482b-483c-8a0e-091a3c6d8399"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Login Word: 12\n",
            "peritungan sigma: 58\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def calculate_word_topic_probability(word_idx, topic_idx, word_length):\n",
        "  ALPHA = 0.1\n",
        "  # Perhitungan Sigma words (topic: 1 -> idx: 0)\n",
        "  sigma_opt_total = 0\n",
        "  for word_topic_matrix in words_topic_matrix :\n",
        "    sigma_opt_total += word_topic_matrix[0]\n",
        "\n",
        "  # Word Topic value\n",
        "  word_topic_val = words_topic_matrix[word_idx][topic_idx]\n",
        "\n",
        "  return (word_topic_val + ALPHA) / (sigma_opt_total + word_length * ALPHA)"
      ],
      "metadata": {
        "id": "C1y6vwiW9GUK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h4><i><b>Simulasi :</b></i></h4>"
      ],
      "metadata": {
        "id": "twLOTZbE922s"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Login (word: 2 -> idx: 1; topic: 1 -> idx: 0)\n",
        "word_idx = 1\n",
        "\n",
        "# Perhitungan Sigma words (topic: 1 -> idx: 0)\n",
        "topic_idx = 0\n",
        "\n",
        "for k_word, v_word in enumerate(tokenize):\n",
        "  probability = calculate_word_topic_probability(\n",
        "      word_idx = word_idx,\n",
        "      topic_idx = topic_idx,\n",
        "      word_length = len(v_word),\n",
        "  )\n",
        "  print(\n",
        "      f\"[ Word {k_word + 1} (Login) ]\\n\" +\n",
        "      f\"word_idx: {word_idx}\\n\" +\n",
        "      f\"topic_idx: {topic_idx}\\n\" +\n",
        "      f\"word_length: {len(v_word)}\\n\" +\n",
        "      f\"Total Topic: {topic_total}\\n\" +\n",
        "      f\"{'-'*10}\\nProbablility: {probability}\\n\\n\"\n",
        "  )\n",
        "\n",
        "  calculate_word_topic_probability_result = probability"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "09tlMGAsBpUq",
        "outputId": "7aea34fe-0d6a-4022-fb13-60b75e36c00e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[ Word 1 (Login) ]\n",
            "word_idx: 1\n",
            "topic_idx: 0\n",
            "word_length: 6\n",
            "Total Topic: 3\n",
            "----------\n",
            "Probablility: 0.20648464163822525\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Tahapan terakhir adalah menetapkan w ke topik baru dimana topik tersebut ditentukan dengan rumus probabilitas $P(t_k\\space| D_i)$ * $P(w_j\\space| T_k)$. Sehingga perhitungannya adalah sebagai berikut :</h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "-ThhIjhbO_L-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "probability = calculate_topic_document_probability_result * calculate_word_topic_probability_result\n",
        "print(f\"probability: {probability}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Dmmi5lkjFL2W",
        "outputId": "77a7cfc3-23cb-448b-f5a1-7564248ead26"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "probability: 0.042855302981518455\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Menghitung probabilitas distribusi co-occurrence $P(W1, W2 | T) = Count(W1, W2, T) / Count(W1, T)$</h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "NN7GKaDrQKTz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def cooccurrence(word1_probability, word2_probability):\n",
        "  return (word1_probability * word2_probability) / word1_probability"
      ],
      "metadata": {
        "id": "0xMY3mSVQJSZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Simulasi mencari $PMI$ dari kata <i><b>\"cant login\"</b></i> dan <i><b>\"change password\"</b></i></h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "p-TBNm-DR5i3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from math import log\n",
        "\n",
        "# \"cant login\" word\n",
        "cant_prob  = calculate_word_topic_probability(\n",
        "      word_idx = 0,\n",
        "      topic_idx = topic_idx,\n",
        "      word_length = len(tokenize[0]),\n",
        ")\n",
        "login_prob = calculate_word_topic_probability_result\n",
        "cant_login_cooccurance = cooccurrence(cant_prob, login_prob)\n",
        "cant_login_pmi = log(cant_login_cooccurance / (cant_prob * login_prob))\n",
        "print(\n",
        "    f\"word: 'cant login'\\n\" +\n",
        "    f\"cooccurance: {cant_login_cooccurance}\\n\" +\n",
        "    f\"pmi: {cant_login_pmi}\\n\\n\"\n",
        ")\n",
        "\n",
        "\n",
        "# \"change password\" word\n",
        "change_prob = calculate_word_topic_probability(\n",
        "      word_idx = 4,\n",
        "      topic_idx = topic_idx,\n",
        "      word_length = len(tokenize[0]),\n",
        ")\n",
        "password_prob = calculate_word_topic_probability(\n",
        "      word_idx = 5,\n",
        "      topic_idx = topic_idx,\n",
        "      word_length = len(tokenize[0]),\n",
        ")\n",
        "change_password_cooccurance = cooccurrence(change_prob, password_prob)\n",
        "change_password_pmi = log(change_password_cooccurance / (change_prob * password_prob))\n",
        "print(\n",
        "    f\"word: 'change password'\\n\" +\n",
        "    f\"cooccurance: {change_password_cooccurance}\\n\" +\n",
        "    f\"pmi: {change_password_pmi}\\n\\n\"\n",
        ")\n",
        "\n",
        "coherence_datas = [cant_login_pmi, change_password_pmi]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_DxpI5ZZS4hG",
        "outputId": "116da08b-0468-473f-9111-4dc43d659daa"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "word: 'cant login'\n",
            "cooccurance: 0.20648464163822525\n",
            "pmi: 2.110639912535697\n",
            "\n",
            "\n",
            "word: 'change password'\n",
            "cooccurance: 0.10409556313993173\n",
            "pmi: 1.8624602830601626\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<br>\n",
        "<ul>\n",
        "  <li><h4>Simulasi menghitung Coherence Score</h4></li>\n",
        "</ul>"
      ],
      "metadata": {
        "id": "MgrxWbWbV0V_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "coherence_score = sum(coherence_datas) / len(coherence_datas)\n",
        "print(f\"Coherence Score: {coherence_score}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N7X8eCZwWImE",
        "outputId": "381ce8f0-0ea8-4158-f398-51f900151476"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Coherence Score: 1.98655009779793\n"
          ]
        }
      ]
    }
  ]
}