diff --git a/FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners_v2-2.ipynb b/FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners_v2-2.ipynb index e6d7b00..212b5b9 100644 --- a/FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners_v2-2.ipynb +++ b/FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners_v2-2.ipynb @@ -2,40 +2,31 @@ "cells": [ { "cell_type": "markdown", + "metadata": { + "id": "X8H-Vc6w6WSU" + }, "source": [ "# Getting Started with FinGPT\n", "Welcome to this comprehensive guide aimed at beginners diving into the realm of Financial Large Language Models (FinLLMs) with FinGPT. This blog post demystifies the process of training FinGPT using Low-Rank Adaptation (LoRA) with the robust base model ChatGlm2-6b.\n", "\n" - ], - "metadata": { - "id": "X8H-Vc6w6WSU" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "4oLjc7bbv0tO" + }, "source": [ "## Part 1: Preparing the Data\n", "Data preparation is a crucial step when it comes to training Financial Large Language Models. Here, we’ll guide you on how to get your dataset ready for FinGPT using Python.\n", "\n", "In this section, you’ve initialized your working directory and loaded a financial sentiment dataset. Let’s break down the steps:\n", "\n" - ], - "metadata": { - "id": "4oLjc7bbv0tO" - } + ] }, { "cell_type": "code", - "source": [ - "!pip install datasets transformers torch tqdm pandas huggingface_hub\n", - "!pip install sentencepiece\n", - "!pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate\n", - "!pip install torchvision torchaudio\n", - "!pip install loguru\n", - "!pip install datasets\n", - "!pip install peft\n", - "!pip install tensorboard" - ], + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -43,11 +34,10 @@ "id": "-maUV8CH7JPB", "outputId": "edda5958-0427-4aa7-9b8f-ec83f66de62d" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.18.0)\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.30.2)\n", @@ -225,18 +215,28 @@ "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard) (3.2.2)\n" ] } + ], + "source": [ + "!pip install datasets transformers torch tqdm pandas huggingface_hub\n", + "!pip install sentencepiece\n", + "!pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate\n", + "!pip install torchvision torchaudio\n", + "!pip install loguru\n", + "!pip install datasets\n", + "!pip install peft\n", + "!pip install tensorboard" ] }, { "cell_type": "markdown", + "metadata": { + "id": "hJp_UOiB70o3" + }, "source": [ "### 1.1 Initialize Directories:\n", "This block checks if certain paths exist; if they do, it deletes them to avoid data conflicts, and then creates a new directory for the upcoming data.\n", "\n" - ], - "metadata": { - "id": "hJp_UOiB70o3" - } + ] }, { "cell_type": "code", @@ -293,8 +293,8 @@ }, "outputs": [ { - "output_type": "stream", "name": "stderr", + "output_type": "stream", "text": [ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n", "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", @@ -305,7 +305,6 @@ ] }, { - "output_type": "execute_result", "data": { "text/plain": [ "Dataset({\n", @@ -314,8 +313,9 @@ "})" ] }, + "execution_count": 3, "metadata": {}, - "execution_count": 3 + "output_type": "execute_result" } ], "source": [ @@ -340,12 +340,12 @@ }, { "cell_type": "markdown", - "source": [ - "### 1.3 Concatenate and Shuffle Dataset" - ], "metadata": { "id": "BRXfrEc8-yWy" - } + }, + "source": [ + "### 1.3 Concatenate and Shuffle Dataset" + ] }, { "cell_type": "code", @@ -359,21 +359,21 @@ }, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "19086\n" ] }, { - "output_type": "execute_result", "data": { "text/plain": [ "(19086, 3)" ] }, + "execution_count": 4, "metadata": {}, - "execution_count": 4 + "output_type": "execute_result" } ], "source": [ @@ -387,12 +387,12 @@ }, { "cell_type": "markdown", - "source": [ - "Now that your training data is loaded and prepared." - ], "metadata": { "id": "oOVsy4if8_et" - } + }, + "source": [ + "Now that your training data is loaded and prepared." + ] }, { "cell_type": "markdown", @@ -407,14 +407,14 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "1vI4We-r9QXh" + }, "source": [ "### 2.1 Dataset Formatting:\n", "You need to structure your data in a specific format that aligns with the training process.\n", "\n" - ], - "metadata": { - "id": "1vI4We-r9QXh" - } + ] }, { "cell_type": "code", @@ -468,6 +468,7 @@ "metadata": { "colab": { "base_uri": "https://localhost:8080/", + "height": 49, "referenced_widgets": [ "8374cb49ce5a49c9a5fd34c7b7c39bd8", "dfca8e8ee6ed4e9ba4ca71fae03a0b91", @@ -480,26 +481,25 @@ "ce62ecf7cbac4ce489844d241c5298e7", "ce798bad2e2f44e9a4b0fb954da37617", "93dba17d279a4b908a101b24a7e2dc0f" - ], - "height": 49 + ] }, "id": "VuuRhlD1pjqH", "outputId": "b14c59a7-d2b1-4d58-b2f2-144c6b395698" }, "outputs": [ { - "output_type": "display_data", "data": { - "text/plain": [ - "formatting..: 0%| | 0/19086 [00:00=1.17 in /usr/local/lib/python3.10/dist-packages (from peft) (1.25.2)\n", @@ -733,18 +730,21 @@ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.13.0->peft) (1.3.0)\n" ] } + ], + "source": [ + "pip install --upgrade peft\n" ] }, { "cell_type": "markdown", + "metadata": { + "id": "aLjcEr3FBGzi" + }, "source": [ "### 3.1 Training Arguments Setup:\n", "Initialize and set training arguments.\n", "\n" - ], - "metadata": { - "id": "aLjcEr3FBGzi" - } + ] }, { "cell_type": "code", @@ -805,9 +805,7 @@ }, { "cell_type": "code", - "source": [ - "!pip list | grep bitsandbytes" - ], + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -815,45 +813,44 @@ "id": "balVgyAv_dGE", "outputId": "630cf4f8-7641-4a71-9ce7-8ff216ebde53" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "bitsandbytes 0.43.0\n" ] } + ], + "source": [ + "!pip list | grep bitsandbytes" ] }, { "cell_type": "markdown", + "metadata": { + "id": "QGabm7cyBM6r" + }, "source": [ "### 3.2 Quantization Config Setup:\n", "Set quantization configuration to reduce model size without losing significant precision.\n", "\n" - ], - "metadata": { - "id": "QGabm7cyBM6r" - } + ] }, { "cell_type": "code", - "source": [ - "!pip install -i https://test.pypi.org/simple/ bitsandbytes" - ], + "execution_count": null, "metadata": { - "id": "Sjs_V1ESicbi", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "Sjs_V1ESicbi", "outputId": "4e543409-746e-4345-9689-6e31c8d5b210" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Looking in indexes: https://test.pypi.org/simple/\n", "Requirement already satisfied: bitsandbytes in /usr/local/lib/python3.10/dist-packages (0.43.0)\n", @@ -882,6 +879,19 @@ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->bitsandbytes) (1.3.0)\n" ] } + ], + "source": [ + "!pip install -i https://test.pypi.org/simple/ bitsandbytes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Upgrade bitsandbytes so the notebook is using version 0.43.0 (note that at the time of writing this is a development version but what the initial notepad is based off of)\n", + "!pip install --upgrade bitsandbytes" ] }, { @@ -902,21 +912,18 @@ }, { "cell_type": "code", - "source": [ - "!pip install accelerate" - ], + "execution_count": null, "metadata": { - "id": "ApgMsCAPilLQ", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "ApgMsCAPilLQ", "outputId": "9672acff-44f8-4dba-c999-13287e45298d" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.28.0)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.25.2)\n", @@ -955,38 +962,40 @@ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n" ] } + ], + "source": [ + "!pip install accelerate" ] }, { "cell_type": "code", - "source": [ - "import os\n", - "os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'\n" - ], + "execution_count": null, "metadata": { "id": "CYNHL7xzGhBC" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "import os\n", + "os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'\n" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "WMqFjp_mBVqO" + }, "source": [ "### 3.3 Model Loading & Preparation:\n", "Load the base model and tokenizer, and prepare the model for INT8 training.\n", "\n", "* **Runtime -> Change runtime type -> A100 GPU**\n", "* retart runtime and run again if not working\n" - ], - "metadata": { - "id": "WMqFjp_mBVqO" - } + ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "th_3Rnqy9Rkg", "colab": { "base_uri": "https://localhost:8080/", "height": 49, @@ -1004,22 +1013,23 @@ "8a085b2c42914ef88ef615093ee2ef06" ] }, + "id": "th_3Rnqy9Rkg", "outputId": "f15b3c22-60d5-42fb-838c-a84bd910d54b" }, "outputs": [ { - "output_type": "display_data", "data": { - "text/plain": [ - "Loading checkpoint shards: 0%| | 0/7 [00:00 to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is\n", ":DefaultFlowCallback\n", @@ -1347,11 +1357,7 @@ ] }, { - "output_type": "display_data", "data": { - "text/plain": [ - "" - ], "text/html": [ "\n", "
\n", @@ -1375,9 +1381,13 @@ " \n", " \n", "

" + ], + "text/plain": [ + "" ] }, - "metadata": {} + "metadata": {}, + "output_type": "display_data" } ], "source": [ @@ -1400,32 +1410,29 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "brHTWfnmCn5D" + }, "source": [ "### 4.3 Model Saving and Download:\n", "After training, save and download your model. You can also check the model's size.\n", "\n" - ], - "metadata": { - "id": "brHTWfnmCn5D" - } + ] }, { "cell_type": "code", - "source": [ - "!zip -r /content/saved_model.zip /content/{training_args.output_dir}\n" - ], + "execution_count": null, "metadata": { - "id": "HUYxzwS_9lMI", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "HUYxzwS_9lMI", "outputId": "13466899-3c26-4812-9a4d-cdc741366b66" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ " adding: content/./finetuned_model/ (stored 0%)\n", " adding: content/./finetuned_model/README.md (deflated 66%)\n", @@ -1443,142 +1450,101 @@ " adding: content/./finetuned_model/adapter_model.safetensors (deflated 7%)\n" ] } + ], + "source": [ + "!zip -r /content/saved_model.zip /content/{training_args.output_dir}\n" ] }, { "cell_type": "code", - "source": [ - "# download to local\n", - "from google.colab import files\n", - "files.download('/content/saved_model.zip')" - ], + "execution_count": null, "metadata": { - "id": "s-7s2Cjw9pAM", "colab": { "base_uri": "https://localhost:8080/", "height": 17 }, + "id": "s-7s2Cjw9pAM", "outputId": "736ce0ae-cc97-41c4-906d-24b70d4cb005" }, - "execution_count": null, "outputs": [ { - "output_type": "display_data", "data": { + "application/javascript": "\n async function download(id, filename, size) {\n if (!google.colab.kernel.accessAllowed) {\n return;\n }\n const div = document.createElement('div');\n const label = document.createElement('label');\n label.textContent = `Downloading \"${filename}\": `;\n div.appendChild(label);\n const progress = document.createElement('progress');\n progress.max = size;\n div.appendChild(progress);\n document.body.appendChild(div);\n\n const buffers = [];\n let downloaded = 0;\n\n const channel = await google.colab.kernel.comms.open(id);\n // Send a message to notify the kernel that we're ready.\n channel.send({})\n\n for await (const message of channel.messages) {\n // Send a message to notify the kernel that we're ready.\n channel.send({})\n if (message.buffers) {\n for (const buffer of message.buffers) {\n buffers.push(buffer);\n downloaded += buffer.byteLength;\n progress.value = downloaded;\n }\n }\n }\n const blob = new Blob(buffers, {type: 'application/binary'});\n const a = document.createElement('a');\n a.href = window.URL.createObjectURL(blob);\n a.download = filename;\n div.appendChild(a);\n a.click();\n div.remove();\n }\n ", "text/plain": [ "" - ], - "application/javascript": [ - "\n", - " async function download(id, filename, size) {\n", - " if (!google.colab.kernel.accessAllowed) {\n", - " return;\n", - " }\n", - " const div = document.createElement('div');\n", - " const label = document.createElement('label');\n", - " label.textContent = `Downloading \"${filename}\": `;\n", - " div.appendChild(label);\n", - " const progress = document.createElement('progress');\n", - " progress.max = size;\n", - " div.appendChild(progress);\n", - " document.body.appendChild(div);\n", - "\n", - " const buffers = [];\n", - " let downloaded = 0;\n", - "\n", - " const channel = await google.colab.kernel.comms.open(id);\n", - " // Send a message to notify the kernel that we're ready.\n", - " channel.send({})\n", - "\n", - " for await (const message of channel.messages) {\n", - " // Send a message to notify the kernel that we're ready.\n", - " channel.send({})\n", - " if (message.buffers) {\n", - " for (const buffer of message.buffers) {\n", - " buffers.push(buffer);\n", - " downloaded += buffer.byteLength;\n", - " progress.value = downloaded;\n", - " }\n", - " }\n", - " }\n", - " const blob = new Blob(buffers, {type: 'application/binary'});\n", - " const a = document.createElement('a');\n", - " a.href = window.URL.createObjectURL(blob);\n", - " a.download = filename;\n", - " div.appendChild(a);\n", - " a.click();\n", - " div.remove();\n", - " }\n", - " " ] }, - "metadata": {} + "metadata": {}, + "output_type": "display_data" }, { - "output_type": "display_data", "data": { + "application/javascript": "download(\"download_4d446262-a30c-4c47-8f7b-7d2861d99bbf\", \"saved_model.zip\", 28952113)", "text/plain": [ "" - ], - "application/javascript": [ - "download(\"download_4d446262-a30c-4c47-8f7b-7d2861d99bbf\", \"saved_model.zip\", 28952113)" ] }, - "metadata": {} + "metadata": {}, + "output_type": "display_data" } + ], + "source": [ + "# download to local\n", + "from google.colab import files\n", + "files.download('/content/saved_model.zip')" ] }, { "cell_type": "code", - "source": [ - "# save to google drive\n", - "from google.colab import drive\n", - "drive.mount('/content/drive')" - ], + "execution_count": null, "metadata": { - "id": "rvBgMgO8RADU", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "rvBgMgO8RADU", "outputId": "02ce899f-d833-4ac2-f4a2-aa93458dfad3" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Mounted at /content/drive\n" ] } + ], + "source": [ + "# save to google drive\n", + "from google.colab import drive\n", + "drive.mount('/content/drive')" ] }, { "cell_type": "code", - "source": [ - "# save the finetuned model to google drive\n", - "!cp -r \"/content/finetuned_model\" \"/content/drive/MyDrive\"\n" - ], + "execution_count": null, "metadata": { "id": "UUctmjm8RIfQ" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "# save the finetuned model to google drive\n", + "!cp -r \"/content/finetuned_model\" \"/content/drive/MyDrive\"\n" + ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "unRoLshR9RQZ", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "unRoLshR9RQZ", "outputId": "540ef5b7-16e2-412e-d350-cc0ff64ef8c4" }, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Model size: 29.84132957458496 MB\n" ] @@ -1599,51 +1565,42 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "1LCjYKuoCusU" + }, "source": [ "Now your model is trained and saved! You can download it and use it for generating financial insights or any other relevant tasks in the finance domain. The usage of TensorBoard allows you to deeply understand and visualize the training dynamics and performance of your model in real-time.\n", "\n", "Happy FinGPT Training! 🚀" - ], - "metadata": { - "id": "1LCjYKuoCusU" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "76g_Qlp8t_Yp" + }, "source": [ "## Part 5: Inference and Benchmarks using FinGPT\n", "Now that your model is trained, let’s understand how to use it to infer and run benchmarks.\n", "* Took about 10 compute units\n", "\n" - ], - "metadata": { - "id": "76g_Qlp8t_Yp" - } + ] }, { "cell_type": "code", - "source": [ - "!pip install transformers==4.30.2 peft==0.4.0\n", - "!pip install sentencepiece\n", - "!pip install accelerate\n", - "!pip install torch\n", - "!pip install peft\n", - "!pip install datasets\n", - "!pip install bitsandbytes" - ], + "execution_count": null, "metadata": { - "id": "ehjG2bpft_OH", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, + "id": "ehjG2bpft_OH", "outputId": "44c97849-bbc6-420d-87a5-0215cfc75ed8" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Requirement already satisfied: transformers==4.30.2 in /usr/local/lib/python3.10/dist-packages (4.30.2)\n", "Collecting peft==0.4.0\n", @@ -1695,22 +1652,22 @@ ] }, { - "output_type": "display_data", "data": { "application/vnd.colab-display-data+json": { + "id": "d8e1a8c74ed44abea6d078426271741d", "pip_warning": { "packages": [ "peft" ] - }, - "id": "d8e1a8c74ed44abea6d078426271741d" + } } }, - "metadata": {} + "metadata": {}, + "output_type": "display_data" }, { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (0.1.99)\n", "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.28.0)\n", @@ -1865,39 +1822,40 @@ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->bitsandbytes) (1.3.0)\n" ] } + ], + "source": [ + "!pip install transformers==4.30.2 peft==0.4.0\n", + "!pip install sentencepiece\n", + "!pip install accelerate\n", + "!pip install torch\n", + "!pip install peft\n", + "!pip install datasets\n", + "!pip install bitsandbytes" ] }, { "cell_type": "markdown", - "source": [ - "### 5.1 Load the model" - ], "metadata": { "id": "P91SXTrLS34i" - } + }, + "source": [ + "### 5.1 Load the model" + ] }, { "cell_type": "code", - "source": [ - "#clone the FinNLP repository\n", - "!git clone https://github.com/AI4Finance-Foundation/FinNLP.git\n", - "\n", - "\n", - "import sys\n", - "sys.path.append('/content/FinNLP/')" - ], + "execution_count": null, "metadata": { - "id": "y5jyY7S_uEls", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "y5jyY7S_uEls", "outputId": "a46b8eb9-2098-4fd3-c7c0-38c2e8263575" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Cloning into 'FinNLP'...\n", "remote: Enumerating objects: 1412, done.\u001b[K\n", @@ -1908,10 +1866,23 @@ "Resolving deltas: 100% (617/617), done.\n" ] } + ], + "source": [ + "#clone the FinNLP repository\n", + "!git clone https://github.com/AI4Finance-Foundation/FinNLP.git\n", + "\n", + "\n", + "import sys\n", + "sys.path.append('/content/FinNLP/')" ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "zRsmSTFZuEjt" + }, + "outputs": [], "source": [ "from transformers import AutoModel, AutoTokenizer, AutoModelForCausalLM\n", "\n", @@ -1923,31 +1894,23 @@ "from finnlp.benchmarks.fiqa import test_fiqa , add_instructions\n", "from finnlp.benchmarks.tfns import test_tfns\n", "from finnlp.benchmarks.nwgi import test_nwgi" - ], - "metadata": { - "id": "zRsmSTFZuEjt" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", - "source": [ - "!pip install --upgrade peft" - ], + "execution_count": null, "metadata": { - "id": "EBqKeUYV9VjF", "colab": { "base_uri": "https://localhost:8080/", "height": 932 }, + "id": "EBqKeUYV9VjF", "outputId": "b342819b-46e6-4972-f0ed-8619c5f80db6" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Requirement already satisfied: peft in /usr/local/lib/python3.10/dist-packages (0.4.0)\n", "Collecting peft\n", @@ -1999,103 +1962,94 @@ ] }, { - "output_type": "display_data", "data": { "application/vnd.colab-display-data+json": { + "id": "b05232126dff43bda21cf143d8fe9952", "pip_warning": { "packages": [ "peft" ] - }, - "id": "b05232126dff43bda21cf143d8fe9952" + } } }, - "metadata": {} + "metadata": {}, + "output_type": "display_data" } + ], + "source": [ + "!pip install --upgrade peft" ] }, { "cell_type": "code", - "source": [ - "# load model from google drive\n", - "from google.colab import drive\n", - "drive.mount('/content/drive')" - ], + "execution_count": null, "metadata": { - "id": "gRRw9drdA2hv", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "gRRw9drdA2hv", "outputId": "c97a6c0e-ef19-4aac-86bd-66f9659e8e34" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n" ] } + ], + "source": [ + "# load model from google drive\n", + "from google.colab import drive\n", + "drive.mount('/content/drive')" ] }, { "cell_type": "code", - "source": [ - "# Define the path you want to check\n", - "path_to_check = \"/content/drive/My Drive/finetuned_model\"\n", - "\n", - "# Check if the specified path exists\n", - "if os.path.exists(path_to_check):\n", - " print(\"Path exists.\")\n", - "else:\n", - " print(\"Path does not exist.\")\n" - ], + "execution_count": null, "metadata": { - "id": "LxCAhg9QpkyI", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "LxCAhg9QpkyI", "outputId": "b135199e-13e1-4149-c0a7-3f4c8267c02b" }, - "execution_count": null, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Path exists.\n" ] } + ], + "source": [ + "# Define the path you want to check\n", + "path_to_check = \"/content/drive/My Drive/finetuned_model\"\n", + "\n", + "# Check if the specified path exists\n", + "if os.path.exists(path_to_check):\n", + " print(\"Path exists.\")\n", + "else:\n", + " print(\"Path does not exist.\")\n" ] }, { "cell_type": "code", - "source": [ - "from transformers import AutoModelForSequenceClassification\n" - ], + "execution_count": null, "metadata": { "id": "UNgGYBd6Piqm" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "from transformers import AutoModelForSequenceClassification\n" + ] }, { "cell_type": "code", - "source": [ - "## load the chatglm2-6b base model\n", - "base_model = \"THUDM/chatglm2-6b\"\n", - "peft_model = training_args.output_dir\n", - "\n", - "tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)\n", - "model = AutoModel.from_pretrained(base_model, trust_remote_code=True, load_in_8bit=True, device_map = \"cuda:0\")\n", - "\n", - "model = PeftModel.from_pretrained(model, peft_model)\n", - "\n", - "model = model.eval()" - ], + "execution_count": null, "metadata": { - "id": "bRljPCKC_srt", "colab": { "base_uri": "https://localhost:8080/", "height": 49, @@ -2113,42 +2067,42 @@ "ba1034b0e14047929f837905572dca87" ] }, + "id": "bRljPCKC_srt", "outputId": "438f4c2f-7a92-4395-f508-aa9dfc65123d" }, - "execution_count": null, "outputs": [ { - "output_type": "display_data", "data": { - "text/plain": [ - "Loading checkpoint shards: 0%| | 0/7 [00:00