FairVIC_Recommend_Lambda_Code.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "myeA3QRk9I3N"
      },
      "source": [
        "# FairVIC Supplementary Code\n",
        "In this notebook you can find all of the supplementary code behind our method, FairVIC with the extension to auto tune the Lambda weights.\n",
        "\n",
        "This process can be unstable in terms of model performance, so once it has recommended weights, we suggest applying them to a new model without the extension to ensure maximum performance"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EfZcEn5o9XTh"
      },
      "source": [
        "## Misc (Imports, Misc. functions, preprocessing)\n",
        "This section is full of various imports, generic functions, or preprocessing steps for the data that is used. It is unimportant but skim through if you would like."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "io-pDBGv_T_H"
      },
      "source": [
        "### Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Shw9dLuK8hS3"
      },
      "outputs": [],
      "source": [
        "!pip install ucimlrepo\n",
        "!pip install aif360"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VOzWjMQx_Ob7"
      },
      "outputs": [],
      "source": [
        "# initialisations\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "sns.set(\n",
        "    palette=\"Paired\",\n",
        "    #style='whitegrid',\n",
        "    color_codes=True,\n",
        "    rc={\"figure.figsize\": (12,8)}\n",
        ")\n",
        "\n",
        "# fairness metrics\n",
        "from aif360.datasets import BinaryLabelDataset\n",
        "from aif360.metrics import BinaryLabelDatasetMetric, ClassificationMetric\n",
        "\n",
        "# model building\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score, precision_score, recall_score, f1_score\n",
        "\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras import Sequential\n",
        "from tensorflow.keras.layers import Dense, Dropout, BatchNormalization, Activation, Dropout\n",
        "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau\n",
        "from tensorflow.keras.optimizers import Adam\n",
        "from tensorflow.keras.regularizers import L2\n",
        "from tensorflow.keras.metrics import AUC, Precision, Recall, F1Score\n",
        "from tensorflow.keras.losses import binary_crossentropy\n",
        "\n",
        "# save results\n",
        "import pickle\n",
        "\n",
        "# suppress warnings\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "import logging\n",
        "logging.getLogger('matplotlib.font_manager').disabled = True"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "__moCeRi_V1g"
      },
      "source": [
        "### Import the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "301zwZru_PDz"
      },
      "outputs": [],
      "source": [
        "from ucimlrepo import fetch_ucirepo\n",
        "\n",
        "# fetch dataset\n",
        "adult = fetch_ucirepo(id=2)\n",
        "\n",
        "# data (as pandas dataframes)\n",
        "X = adult.data.features\n",
        "y = adult.data.targets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SZrvsERY_Pv7"
      },
      "outputs": [],
      "source": [
        "# check to see if the data has imported correctly\n",
        "X.head(4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ubovq8Gf_P8i"
      },
      "outputs": [],
      "source": [
        "# sort the target variables to be correct\n",
        "y['income'] = y['income'].replace({'<=50K.': '<=50K', '>50K.': '>50K'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4pIop-MT_1kd"
      },
      "source": [
        "### Preprocessing data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KSmMTxxG_QIy"
      },
      "outputs": [],
      "source": [
        "# feature selection\n",
        "X = X[['age', 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'capital-gain', 'capital-loss', 'hours-per-week', 'native-country']]\n",
        "X.dtypes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oE5ogWlQ_QYj"
      },
      "outputs": [],
      "source": [
        "X['workclass'] = X['workclass'].astype('category').cat.codes\n",
        "X['education'] = X['education'].astype('category').cat.codes\n",
        "X['marital-status'] = X['marital-status'].astype('category').cat.codes\n",
        "X['occupation'] = X['occupation'].astype('category').cat.codes\n",
        "X['relationship'] = X['relationship'].astype('category').cat.codes\n",
        "X['race'] = X['race'].astype('category').cat.codes\n",
        "X['sex'] = X['sex'].astype('category').cat.codes\n",
        "X['native-country'] = X['native-country'].astype('category').cat.codes\n",
        "y['income'] = y['income'].astype('category').cat.codes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MV0B0vJY_QkC"
      },
      "outputs": [],
      "source": [
        "X = X.astype('float32')\n",
        "y = y.astype('float32')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RZYXDI4P_QvZ"
      },
      "outputs": [],
      "source": [
        "y = y['income']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H8mEVyTn9rDQ"
      },
      "source": [
        "## FairVIC\n",
        "Here you will find the code behind our method, FairVIC, please read throught the comments for explanations 😀"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "afs0qc8KC-9Z"
      },
      "source": [
        "### FairVIC class\n",
        "Here is all the code, in one class, for training a custom neural network on our new loss function FairVIC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DKZj9W9y9pa_"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "Our fairness loss function, FairVIC\n",
        "Returns:\n",
        "  trained model -> a trained model that aims to be as fair as possible\n",
        "\"\"\"\n",
        "class FairModel():\n",
        "  # standard initialisations\n",
        "  def __init__(self):\n",
        "    self.model = self.create_model()\n",
        "    self.protected_attribute = None\n",
        "    self.privileged_groups = None\n",
        "    self.unprivileged_groups = None\n",
        "    self.favorable_label = None\n",
        "    self.unfavorable_label = None\n",
        "\n",
        "    # trainable lambdas params\n",
        "    self.lambda_binary = tf.Variable(0.25, trainable=True, dtype=tf.float32, constraint=lambda x: tf.clip_by_value(x, 0.1, 0.7))\n",
        "    self.lambda_cov = tf.Variable(0.25, trainable=True, dtype=tf.float32, constraint=lambda x: tf.clip_by_value(x, 0.1, 0.7))\n",
        "    self.lambda_var = tf.Variable(0.25, trainable=True, dtype=tf.float32, constraint=lambda x: tf.clip_by_value(x, 0.1, 0.7))\n",
        "    self.lambda_inv = tf.Variable(0.25, trainable=True, dtype=tf.float32, constraint=lambda x: tf.clip_by_value(x, 0.1, 0.7))\n",
        "\n",
        "    # optimizers here instead please\n",
        "    self.optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)\n",
        "    self.lambda_optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)\n",
        "\n",
        "  # define what certain attributes are as to evaulate for fairness\n",
        "  def configure_fairness_evaluation(self, protected_attribute, privileged_groups, unprivileged_groups, favorable_label, unfavorable_label):\n",
        "    self.protected_attribute = protected_attribute\n",
        "    self.privileged_groups = privileged_groups\n",
        "    self.unprivileged_groups = unprivileged_groups\n",
        "    self.favorable_label = favorable_label\n",
        "    self.unfavorable_label = unfavorable_label\n",
        "\n",
        "  # the FairVIC custom loss function\n",
        "  def fairvic_wrapper(self, protected_attribute, eps=1e-4):\n",
        "      def fairvic_loss(y_true, y_pred):\n",
        "        y_pred_squeezed = tf.squeeze(y_pred, axis=1)\n",
        "\n",
        "        # this can be replaced with any accuracy loss, for example purposes, we use binary cross entropy\n",
        "        binary_loss = binary_crossentropy(y_true, y_pred_squeezed)\n",
        "\n",
        "        protected = tf.reshape(protected_attribute, (-1, 1))\n",
        "\n",
        "        # calculate the variance term\n",
        "        variance_loss = tf.reduce_mean(tf.maximum(0., 1. - tf.math.sqrt(tf.reduce_mean(tf.square(protected - tf.reduce_mean(protected, axis=0)), axis=0) + eps)))\n",
        "\n",
        "        # calculate the invariance term\n",
        "        invariance_loss = tf.reduce_mean(tf.square(protected - tf.reduce_mean(protected, axis=0)))\n",
        "\n",
        "        # calculate the covariance term\n",
        "        y_pred_reshaped = tf.reshape(y_pred, (-1, 1))\n",
        "        cov_matrix = tf.matmul(tf.transpose(y_pred_reshaped - tf.reduce_mean(y_pred_reshaped, axis=0)), protected)\n",
        "        covariance_loss = tf.sqrt(tf.reduce_sum(tf.square(cov_matrix))) / tf.cast(tf.shape(y_pred_reshaped)[0], tf.float32)\n",
        "\n",
        "        lambda_regularization = -0.01 * tf.reduce_sum([tf.math.log(self.lambda_binary + 1e-8),\n",
        "                                                       tf.math.log(self.lambda_cov + 1e-8),\n",
        "                                                       tf.math.log(self.lambda_var + 1e-8),\n",
        "                                                       tf.math.log(self.lambda_inv + 1e-8)])\n",
        "\n",
        "        # combining the losses with certain weights\n",
        "        total_loss = (self.lambda_binary * binary_loss +\n",
        "                      self.lambda_var * variance_loss +\n",
        "                      self.lambda_inv * invariance_loss +\n",
        "                      self.lambda_cov * covariance_loss) + lambda_regularization\n",
        "\n",
        "        return total_loss\n",
        "      return fairvic_loss\n",
        "\n",
        "  # define the neural network that will be trained to be fair\n",
        "  def create_model(self):\n",
        "    model = Sequential([\n",
        "        Dense(256, input_shape=(X_train.shape[1],), kernel_regularizer=L2(1e-4), kernel_initializer='he_normal'),\n",
        "        BatchNormalization(),\n",
        "        Activation('relu'),\n",
        "        Dropout(0.25),\n",
        "\n",
        "        Dense(128, kernel_regularizer=L2(1e-4), kernel_initializer='he_normal'),\n",
        "        BatchNormalization(),\n",
        "        Activation('relu'),\n",
        "        Dropout(0.25),\n",
        "\n",
        "        Dense(64, kernel_regularizer=L2(1e-4), kernel_initializer='he_normal'),\n",
        "        BatchNormalization(),\n",
        "        Activation('relu'),\n",
        "        Dropout(0.25),\n",
        "\n",
        "        Dense(32, kernel_regularizer=L2(1e-4), kernel_initializer='he_normal'),\n",
        "        BatchNormalization(),\n",
        "        Activation('relu'),\n",
        "\n",
        "        Dense(1, activation='sigmoid')\n",
        "    ])\n",
        "    return model\n",
        "\n",
        "  # the training loop\n",
        "  @tf.function\n",
        "  def train_step(self, inputs, labels, protected):\n",
        "      with tf.GradientTape() as tape:\n",
        "          predictions = self.model(inputs, training=True)\n",
        "          loss = self.fairvic_wrapper(protected)(labels, predictions)\n",
        "      gradients = tape.gradient(loss, self.model.trainable_variables)\n",
        "      self.optimizer.apply_gradients(zip(gradients, self.model.trainable_variables))\n",
        "\n",
        "      # update lambdas\n",
        "      with tf.GradientTape() as tape_lambdas:\n",
        "          loss_for_lambdas = self.fairvic_wrapper(protected)(labels, predictions)\n",
        "      lambda_gradients = tape_lambdas.gradient(loss_for_lambdas, [self.lambda_binary, self.lambda_cov, self.lambda_var, self.lambda_inv])\n",
        "      clipped_gradients = [tf.clip_by_value(g, -1.0, 1.0) for g in lambda_gradients]\n",
        "      lambda_grad_norms = [tf.norm(g) for g in clipped_gradients]\n",
        "      scaling_factors = [1.0 / (norm + 1e-8) for norm in lambda_grad_norms]\n",
        "      balanced_gradients = [g * scale for g, scale in zip(lambda_gradients, scaling_factors)]\n",
        "      self.lambda_optimizer.apply_gradients(zip(balanced_gradients, [self.lambda_binary, self.lambda_cov, self.lambda_var, self.lambda_inv]))\n",
        "\n",
        "      # normalize lambdas\n",
        "      total_lambda = self.lambda_binary + self.lambda_cov + self.lambda_var + self.lambda_inv\n",
        "      self.lambda_binary.assign((self.lambda_binary / total_lambda))\n",
        "      self.lambda_cov.assign((self.lambda_cov / total_lambda))\n",
        "      self.lambda_var.assign((self.lambda_var / total_lambda))\n",
        "      self.lambda_inv.assign((self.lambda_inv / total_lambda))\n",
        "\n",
        "      return loss\n",
        "\n",
        "  @tf.function\n",
        "  def val_step(self, inputs, labels, protected):\n",
        "      predictions = self.model(inputs, training=False)\n",
        "      loss = self.fairvic_wrapper(protected)(labels, predictions)\n",
        "      return loss\n",
        "\n",
        "\n",
        "  def train(self, epochs, batch_size, X_train, y_train, X_train_protected, X_val, y_val, X_val_protected):\n",
        "      train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train, X_train_protected)).batch(batch_size)\n",
        "      val_dataset = tf.data.Dataset.from_tensor_slices((X_val, y_val, X_val_protected)).batch(batch_size)\n",
        "\n",
        "      best_val_loss = np.inf\n",
        "      epoch_train_losses = []\n",
        "      epoch_val_losses = []\n",
        "\n",
        "      acc_lambda = []\n",
        "      var_lambda = []\n",
        "      cov_lambda = []\n",
        "      inv_lambda = []\n",
        "\n",
        "      for epoch in range(epochs):\n",
        "          print(f\"Epoch {epoch+1}/{epochs}\")\n",
        "          train_losses_batch = []\n",
        "          for batch, (inputs, labels, protected) in enumerate(train_dataset):\n",
        "              loss = self.train_step(inputs, labels, protected)\n",
        "              train_losses_batch.append(loss.numpy())\n",
        "          train_loss = np.mean(train_losses_batch)\n",
        "\n",
        "          val_losses_batch = []\n",
        "          for batch, (inputs, labels, protected) in enumerate(val_dataset):\n",
        "              loss = self.val_step(inputs, labels, protected)\n",
        "              val_losses_batch.append(loss.numpy())\n",
        "          val_loss = np.mean(val_losses_batch)\n",
        "\n",
        "          print(f\"Epoch {epoch+1}, Loss: {train_loss}, Val Loss: {val_loss}, Lr: {self.optimizer.learning_rate.numpy()}\")\n",
        "          epoch_train_losses.append(train_loss)\n",
        "          epoch_val_losses.append(val_loss)\n",
        "\n",
        "          acc_lambda.append(self.lambda_binary.numpy())\n",
        "          var_lambda.append(self.lambda_var.numpy())\n",
        "          cov_lambda.append(self.lambda_cov.numpy())\n",
        "          inv_lambda.append(self.lambda_inv.numpy())\n",
        "\n",
        "      # plot loss (Assuming `plot_losses` is defined)\n",
        "      self.plot_losses(epoch_train_losses, epoch_val_losses)\n",
        "\n",
        "  # plot the loss history for sanity checks\n",
        "  def plot_losses(self, train_losses, val_losses):\n",
        "    plt.title('Learning Curves')\n",
        "    plt.xlabel('Epoch')\n",
        "    plt.ylabel('Loss')\n",
        "    plt.plot(train_losses, label='train')\n",
        "    plt.plot(val_losses, label='val')\n",
        "    plt.legend()\n",
        "    plt.show()\n",
        "\n",
        "  # evaluate the trained model for accuracy metrics\n",
        "  def evaluate(self, X_test, y_test):\n",
        "    predictions = self.model.predict(X_test)\n",
        "    predictions = (predictions > 0.5).astype(int)\n",
        "\n",
        "    acc = accuracy_score(y_test, predictions)\n",
        "    auc = roc_auc_score(y_test, predictions)\n",
        "    precision = precision_score(y_test, predictions)\n",
        "    recall = recall_score(y_test, predictions)\n",
        "    f1 = f1_score(y_test, predictions)\n",
        "\n",
        "    return {'acc': acc, 'auc': auc, 'precision': precision, 'recall': recall, 'f1': f1, 'y_test': y_test, 'y_pred': predictions}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E94F1E9NDCXz"
      },
      "source": [
        "### Using FairVIC\n",
        "Please follow the walkthrough below on how to use FairVIC"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-DaIHCgDXG3"
      },
      "source": [
        "Firstly, we want to split out data in training, validation, and test datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2dNFC3xiDQO2"
      },
      "outputs": [],
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
        "X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PiCJerVSDfut"
      },
      "source": [
        "Next, we want to extract the pritected characteristic column from the dataset. In this psuedo-example, we extract only the 'sex' column"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "L0_Dhe4BDQHV"
      },
      "outputs": [],
      "source": [
        "protected_attribute_column_name = 'sex'\n",
        "X_train_protected = X.loc[X_train.index, protected_attribute_column_name]\n",
        "X_test_protected = X.loc[X_test.index, protected_attribute_column_name]\n",
        "X_val_protected = X.loc[X_val.index, protected_attribute_column_name]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hc-ChxmSDqjH"
      },
      "source": [
        "We then can do some standard scaling of the data which is typical for most neural net training"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uqggUB04DoeA"
      },
      "outputs": [],
      "source": [
        "scaler = MinMaxScaler()\n",
        "X_train = scaler.fit_transform(X_train)\n",
        "X_test = scaler.transform(X_test)\n",
        "X_val = scaler.transform(X_val)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3l4jGP0kD3Nw"
      },
      "source": [
        "Next, we need to define the model. Here we can just call a new instance of the class and it will instantiate a new neural network that has been created using the 'create_model()' function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KEpLjAJRDoVJ"
      },
      "outputs": [],
      "source": [
        "fair_model = FairModel()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cB0Mt76LDoP2"
      },
      "outputs": [],
      "source": [
        "fair_model.train(200, 256,\n",
        "                 X_train, y_train, X_train_protected, X_val, y_val, X_val_protected)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zHZDp2vZE-OG"
      },
      "source": [
        "To evaluate the model's performance in standard accuracy metrics, we have provided a function below to do so. This can obviously be changed to allow for any custom metrics needed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E9svpEecEwxb"
      },
      "outputs": [],
      "source": [
        "eval_metrics = fair_model.evaluate(X_test, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6JhF-Z26FQaF"
      },
      "source": [
        "To evaluate the model's performance in fairness metrics, we use this little code snippet below. All that is needed is to specifiy the 'protected attribute' as well as the 'priviledged' and 'unprivileged' groups."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fKF7qDHlUlum"
      },
      "outputs": [],
      "source": [
        "X_test = pd.DataFrame(data=X_test, columns=['age', 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'capital-gain', 'capital-loss', 'hours-per-week', 'native-country'])\n",
        "predictions = eval_metrics['y_pred']\n",
        "\n",
        "test_df = pd.concat([X_test.reset_index(drop=True), y_test.reset_index(drop=True)], axis=1)\n",
        "test_df.columns = list(X_test.columns) + ['label']\n",
        "\n",
        "test_df = test_df.astype(int)\n",
        "\n",
        "test_bld = BinaryLabelDataset(df=test_df,\n",
        "                              label_names=['label'],\n",
        "                              protected_attribute_names=['sex'],\n",
        "                              favorable_label=1,\n",
        "                              unfavorable_label=0)\n",
        "\n",
        "predictions_bld = test_bld.copy()\n",
        "predictions_bld.labels = predictions.reshape(-1, 1)\n",
        "\n",
        "classification_metric = ClassificationMetric(test_bld,\n",
        "                                              predictions_bld,\n",
        "                                              unprivileged_groups=[{'sex': 0}],\n",
        "                                              privileged_groups=[{'sex': 1}])\n",
        "\n",
        "fairness_metrics = {\n",
        "        'equalized_odds_difference': classification_metric.equalized_odds_difference(),\n",
        "        'average_abs_odds_difference': classification_metric.average_abs_odds_difference(),\n",
        "        'disparate_impact': classification_metric.disparate_impact(),\n",
        "        'demographic_parity_difference': classification_metric.mean_difference(),\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "f4BVkwndodN2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Acc Lambda: {:.2f}\".format(fair_model.lambda_binary.numpy()))\n",
        "print(\"Cov Lambda: {:.2f}\".format(fair_model.lambda_cov.numpy()))\n",
        "print(\"Var Lambda: {:.2f}\".format(fair_model.lambda_var.numpy()))\n",
        "print(\"Inv Lambda: {:.2f}\".format(fair_model.lambda_inv.numpy()))"
      ],
      "metadata": {
        "id": "x9s4e8g1mkvf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oAhLhx2RHxmW"
      },
      "source": [
        "Finally, to see our results for both accuracy and fairness, we print every metric here!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9cmwNYp_Fd9U"
      },
      "outputs": [],
      "source": [
        "print(f\"Results\")\n",
        "print(\"============================================================================================\")\n",
        "print(f\"Accuracy: {eval_metrics['acc']}\")\n",
        "print(f\"AUC: {eval_metrics['auc']}\")\n",
        "print(f\"Precision: {eval_metrics['precision']}\")\n",
        "print(f\"Recall: {eval_metrics['recall']}\")\n",
        "print(f\"F1 Score: {eval_metrics['f1']}\")\n",
        "print(\"---------------------------------------------\")\n",
        "print(\"Fairness Metrics:\")\n",
        "print(f\"Equalized Odds Difference: {fairness_metrics['equalized_odds_difference']}\")\n",
        "print(f\"Average Absolute Odds Difference: {fairness_metrics['average_abs_odds_difference']}\")\n",
        "print(f\"Disparate Impact: {fairness_metrics['disparate_impact']}\")\n",
        "print(f\"Demographic Parity (Mean) Difference: {fairness_metrics['demographic_parity_difference']}\")\n",
        "print(\"============================================================================================\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "auShRw0MH5Al"
      },
      "source": [
        "That is how you can incorporate FairVIC into a neural network's training. It is elegant yet efficient. We encourage you to mess around with the weights! 😀"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "__moCeRi_V1g",
        "4pIop-MT_1kd"
      ],
      "gpuType": "L4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}