6-Vision_Transformer.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Vision Transformer"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this notebook, we are going to fine-tune a pre-trained Vision Transformer (from [🤗 Transformers](https://github.com/huggingface/transformers)) for art classification. We will train the model using [PyTorch Lightning ⚡](https://github.com/PyTorchLightning/pytorch-lightning). \n",
    "\n",
    "HuggingFace 🤗 is a leading open-source software library and community that has gained significant attention in recent years for its contributions to democratizing AI. The library provides pre-trained models, datasets, and a suite of tools that make it easier for developers to build and deploy AI applications. One of the most significant contributions of HuggingFace is the development of the Transformers library, which provides an easy-to-use interface for working with Transformer-based models such as BERT and GPT.\n",
    "\n",
    "PyTorch Lightning is an open-source Python library that provides a high-level interface for PyTorch. This lightweight and high-performance framework organizes PyTorch code to decouple the research from the engineering, making Deep Learning experiments easier to read and reproduce.\n",
    "\n",
    "**Source:** Rogge, N. (2021) [Fine-tuning the Vision Transformer on CIFAR-10 with PyTorch Lightning - GitHub](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/VisionTransformer/Fine_tuning_the_Vision_Transformer_on_CIFAR_10_with_PyTorch_Lightning.ipynb).\n",
    "\n",
    "![vit.png](./docs/Vision_Transformer.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What are Transformers?\n",
    "\n",
    "The Transformer architecture, which was introduced in the seminal paper \"Attention is All You Need\" in 2017, has taken the world of Deep Learning by storm, particularly in the field of Natural Language Processing (NLP). As a large language model first based on the GPT-3.5 architecture, ChatGPT is a prime example of an application based on the Transformer architecture that is now in the public eye. In addition to ChatGPT, many other popular applications, such as Google's BERT, OpenAI's GPT series, and Facebook's RoBERTa, also rely on the Transformer architecture to achieve state-of-the-art results in NLP tasks. Furthermore, the Transformer architecture has also made significant inroads in the field of computer vision, as evidenced by the success of models such as ViT and DeiT on ImageNet and other visual recognition benchmarks.\n",
    "\n",
    "The major innovation of the transformer architecture is combining the use of attention-based representations and a CNN style of processing. Unlike traditional convolutional neural networks (CNNs) that rely on convolutional layers to extract features from images, Transformers use attention mechanisms (self-attention, multi-headed attention) to selectively focus on different parts of an input sequence.\n",
    "\n",
    "The main advantage of Transformers over traditional CNNs is that they can capture long-range dependencies in data more effectively. This is especially useful in Computer Vision tasks where an image may contain objects that are spread out across the image, and where the relationships between objects may be more important than the objects themselves. By attending to different parts of the input image, Transformers can effectively learn to extract these relationships and improve performance on tasks such as object detection and segmentation.\n",
    "\n",
    "\n",
    "**Sources:**\n",
    "\n",
    "+ Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). [Attention is all you need.](https://arxiv.org/abs/1706.03762) - arXiv preprint arXiv:1706.03762. \n",
    "+ Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929) - arXiv preprint arXiv:2010.11929.\n",
    "+ Google Research. (2021). [Vision Transformer and MLP-Mixer Architectures  - GitHub](https://github.com/google-research/vision_transformer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118\n",
    "# !pip install -q transformers datasets pytorch-lightning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\venvs\\no-estruc\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from src.utils import *\n",
    "from pytorch_lightning import Trainer\n",
    "from pytorch_lightning.callbacks import EarlyStopping\n",
    "from src.vit_fine_tune import ViTLightningModule"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our data is downloaded from the above mentioned kaggle URL. We have saved our data in a directory called `data`, which must be located within the same folder as the execution notebooks. Within `data`, we can find two separate directories: `train`, `test` and `validation`, which contain an equal number of randomly selected images from each of the art classes selected. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of classes: 4\n",
      "Existing classes: ['Baroque', 'Realism', 'Renaissance', 'Romanticism']\n",
      "\n",
      "----------------------------------------\n",
      "Number of images per class and dataset:\n",
      "----------------------------------------\n",
      "             Train  Validation  Test\n",
      "Style                               \n",
      "Baroque       4000         500   500\n",
      "Realism       4000         500   500\n",
      "Renaissance   4000         500   500\n",
      "Romanticism   4000         500   500\n"
     ]
    }
   ],
   "source": [
    "data_dir = \"./data\"\n",
    "\n",
    "# Train folder\n",
    "train_dir = os.path.join(data_dir, \"train\")\n",
    "# Validation folder\n",
    "validation_dir = os.path.join(data_dir, \"validation\")\n",
    "# Test folder\n",
    "test_dir = os.path.join(data_dir, \"test\")\n",
    "\n",
    "dataset_stats(train_dir, validation_dir, test_dir)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that all classes are well balanced, and that we have a fair amount of data for training and validation.\n",
    "\n",
    "We now create the directory where the models will be saved:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create directory where to save the models created\n",
    "models_dir = \"./models\"\n",
    "os.makedirs(models_dir, exist_ok=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Activating CUDA for GPU processing"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GPUs (Graphics Processing Units) are specialized processors designed to handle the complex computations involved in rendering graphics and images. However, due to their parallel processing capabilities, they are also useful for a wide range of other applications, including Machine Learning and Scientific Computing. Unlike traditional CPUs (Central Processing Units), which are designed to handle a few tasks very quickly, GPUs can handle many smaller tasks simultaneously, making them ideal for computationally-intensive applications.\n",
    "\n",
    "CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA, designed to harness the power of GPUs for general-purpose computing tasks. CUDA allows developers to write programs that run on the GPU, taking advantage of its parallel processing capabilities to accelerate performance significantly.\n",
    "\n",
    "In order to significantly speed up the training of the model, we will use GPU acceleration. We will first check if CUDA is available in our system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Is CUDA supported by this system? True\n",
      "CUDA version: 11.8\n",
      "ID of current CUDA device: 0\n",
      "Name of current CUDA device: NVIDIA GeForce GTX 1060 with Max-Q Design\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "print(f\"Is CUDA supported by this system? {torch.cuda.is_available()}\")\n",
    "print(f\"CUDA version: {torch.version.cuda}\")\n",
    "  \n",
    "# Storing ID of current CUDA device\n",
    "cuda_id = torch.cuda.current_device()\n",
    "print(f\"ID of current CUDA device: {torch.cuda.current_device()}\")\n",
    "        \n",
    "print(f\"Name of current CUDA device: {torch.cuda.get_device_name(cuda_id)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CUDA is supported by our system, so we will train the models using GPU."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorBoard is a web-based visualization tool provided by TensorFlow for visualizing and analyzing various aspects of machine learning experiments.\n",
    "\n",
    "The %load_ext tensorboard command loads the TensorBoard extension in Jupyter Notebook. %tensorboard --logdir lightning_logs/ command starts TensorBoard and specifies the directory where the logs are stored, in this case `./lightning_logs/` TensorBoard reads the events and metrics logged during the training process and provides visualizations to analyze the model's performance, including loss and accuracy curves, histograms of weights and biases, and more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "      <iframe id=\"tensorboard-frame-aec12220b927e3f0\" width=\"100%\" height=\"800\" frameborder=\"0\">\n",
       "      </iframe>\n",
       "      <script>\n",
       "        (function() {\n",
       "          const frame = document.getElementById(\"tensorboard-frame-aec12220b927e3f0\");\n",
       "          const url = new URL(\"http://localhost\");\n",
       "          const port = 6006;\n",
       "          if (port) {\n",
       "            url.port = port;\n",
       "          }\n",
       "          frame.src = url;\n",
       "        })();\n",
       "      </script>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Start tensorboard.\n",
    "%load_ext tensorboard\n",
    "%tensorboard --logdir lightning_logs/"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use early stopping to stop training when the validation loss stops improving."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at google/vit-base-patch16-224-in21k were not used when initializing ViTForImageClassification: ['pooler.dense.bias', 'pooler.dense.weight']\n",
      "- This IS expected if you are initializing ViTForImageClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing ViTForImageClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of ViTForImageClassification were not initialized from the model checkpoint at google/vit-base-patch16-224-in21k and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "d:\\venvs\\no-estruc\\lib\\site-packages\\pytorch_lightning\\loops\\utilities.py:70: PossibleUserWarning: `max_epochs` was not set. Setting it to 1000 epochs. To train without an epoch limit, set `max_epochs=-1`.\n",
      "  rank_zero_warn(\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "d:\\venvs\\no-estruc\\lib\\site-packages\\transformers\\optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n",
      "\n",
      "  | Name | Type                      | Params\n",
      "---------------------------------------------------\n",
      "0 | vit  | ViTForImageClassification | 85.8 M\n",
      "---------------------------------------------------\n",
      "85.8 M    Trainable params\n",
      "0         Non-trainable params\n",
      "85.8 M    Total params\n",
      "343.207   Total estimated model params size (MB)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 7: 100%|██████████| 1000/1000 [18:37<00:00,  1.12s/it, v_num=7]      \n"
     ]
    }
   ],
   "source": [
    "os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'max_split_size_mb:128'\n",
    "\n",
    "model = ViTLightningModule()\n",
    "\n",
    "early_stop_callback = EarlyStopping(\n",
    "    monitor='val_loss',\n",
    "    patience=3,\n",
    "    strict=False,\n",
    "    verbose=False,\n",
    "    mode='min'\n",
    ")\n",
    "\n",
    "trainer = Trainer(\n",
    "    accelerator='gpu',\n",
    "    devices=1,\n",
    "    callbacks=[\n",
    "        early_stop_callback\n",
    "    ]\n",
    ")\n",
    "\n",
    "trainer.fit(model)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validation"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will first show the test results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\venvs\\no-estruc\\lib\\site-packages\\pytorch_lightning\\trainer\\connectors\\checkpoint_connector.py:148: UserWarning: `.test(ckpt_path=None)` was called without a model. The best model of the previous `fit` call will be used. You can pass `.test(ckpt_path='best')` to use the best model or `.test(ckpt_path='last')` to use the last model. If you pass a value, this warning will be silenced.\n",
      "  rank_zero_warn(\n",
      "Restoring states from the checkpoint path at d:\\Estudios\\Masters\\MBD_ICAI\\Cuatri_2\\NoEstruc\\IMAGES\\Practica\\lightning_logs\\version_7\\checkpoints\\epoch=7-step=8000.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at d:\\Estudios\\Masters\\MBD_ICAI\\Cuatri_2\\NoEstruc\\IMAGES\\Practica\\lightning_logs\\version_7\\checkpoints\\epoch=7-step=8000.ckpt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing DataLoader 0: 100%|██████████| 500/500 [00:56<00:00,  8.83it/s]\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "       Test metric             DataLoader 0\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "        test_loss           0.5024473667144775\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'test_loss': 0.5024473667144775}]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.test()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the reports for the train, test and validation set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at google/vit-base-patch16-224-in21k were not used when initializing ViTForImageClassification: ['pooler.dense.weight', 'pooler.dense.bias']\n",
      "- This IS expected if you are initializing ViTForImageClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing ViTForImageClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of ViTForImageClassification were not initialized from the model checkpoint at google/vit-base-patch16-224-in21k and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using device: cuda\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 88%|████████▊ | 110/125 [15:42<01:47,  7.13s/it]d:\\venvs\\no-estruc\\lib\\site-packages\\PIL\\TiffImagePlugin.py:858: UserWarning: Corrupt EXIF data.  Expecting to read 2 bytes but only got 0. \n",
      "  warnings.warn(str(msg))\n",
      "100%|██████████| 125/125 [17:48<00:00,  8.55s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set classification report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     Baroque       0.95      0.91      0.93      4000\n",
      "     Realism       0.92      0.94      0.93      4000\n",
      " Renaissance       0.96      0.91      0.93      4000\n",
      " Romanticism       0.88      0.94      0.91      4000\n",
      "\n",
      "    accuracy                           0.93     16000\n",
      "   macro avg       0.93      0.93      0.93     16000\n",
      "weighted avg       0.93      0.93      0.93     16000\n",
      "\n",
      "Using device: cuda\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 63/63 [02:26<00:00,  2.32s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set classification report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     Baroque       0.87      0.83      0.85       500\n",
      "     Realism       0.82      0.82      0.82       500\n",
      " Renaissance       0.89      0.85      0.87       500\n",
      " Romanticism       0.74      0.81      0.78       500\n",
      "\n",
      "    accuracy                           0.83      2000\n",
      "   macro avg       0.83      0.83      0.83      2000\n",
      "weighted avg       0.83      0.83      0.83      2000\n",
      "\n",
      "Using device: cuda\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 63/63 [02:24<00:00,  2.29s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation set classification report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "     Baroque       0.89      0.82      0.85       500\n",
      "     Realism       0.83      0.85      0.84       500\n",
      " Renaissance       0.90      0.83      0.86       500\n",
      " Romanticism       0.76      0.86      0.81       500\n",
      "\n",
      "    accuracy                           0.84      2000\n",
      "   macro avg       0.84      0.84      0.84      2000\n",
      "weighted avg       0.84      0.84      0.84      2000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# Load best model from the latest checkpoint\n",
    "best_model = load_latest_checkpoint(ViTLightningModule)\n",
    "# Get best model metrics\n",
    "get_vit_metrics(best_model, train=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Regarding the **training set**, the model exhibits excellent classification metrics such as precision, recall, and F1-score for all the four classes, with scores ranging from 0.88 to 0.96, indicating the model's capability to classify the artworks in the training set with high confidence. The F1-scores for both macro average and weighted average are 0.93, denoting the model's overall good performance, which is also reflected in the high accuracy of 0.93.\n",
    "\n",
    "Moving on to the **test set**, we observe a decrease in the precision, recall, and F1-scores as compared to the training set. Despite the decrease, the model still performs reasonably well with F1-scores varying between 0.74 to 0.89, signifying the model's generalization ability to novel examples. Both macro average and weighted average F1-scores are 0.83, which indicate good performance but slightly lower compared to the training set. However, the model's overall accuracy on the test set is 0.83, which is considerably better than the mentioned models with an accuracy around 60%.\n",
    "\n",
    "Finally, regarding the **validation set**, we observe comparable performance to the test set, with slightly higher F1-scores ranging from 0.76 to 0.90. Both macro average and weighted average F1-scores are 0.84, slightly higher than the performance on the test set.\n",
    "\n",
    "In conclusion, the fine-tuned Vision Transformer model demonstrates high-quality performance for the art classification task, as evidenced by its ability to classify artworks accurately with high precision and recall on the training set. The model's good performance generalizes to new examples with its ability to classify artworks with reasonably high F1-scores on the test and validation sets."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We save the final model in `vit_model.pt`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the model in the models directory\n",
    "torch.save(best_model.state_dict(), os.path.join(models_dir, \"vit_model.pt\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In summary, the Vision Transformer outperforms the previous models by a significant margin. It achieves an accuracy that is approximately 0.2 higher than the other models, representing a 25% increase in performance. The superior performance of the Vision Transformer model can be attributed to its ability to capture global dependencies and interactions between the features. While traditional CNN models rely on convolutional and pooling operations to extract local features and flatten them into a vector, transformers use self-attention mechanisms that enable global interactions among all the features. This allows transformers to model complex relationships between the features and identify long-range dependencies, making them particularly effective for tasks such as image classification.\n",
    "\n",
    "In addition, the hierarchical architecture of the Vision Transformer may also contribute to its success in the art style classification task. The architecture allows it to process images at multiple levels of granularity, from local features to the entire image. This enables the model to learn representations that are better suited for tasks such as image classification, and could explain its strong performance in this project."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "no-estruc",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "2818b84fe0d8ee9ed89d361455090ef436eee20ee147624d1f156870c67bd555"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}