add unsloth ipynb

integrations/model-training/unsloth/notebooks/Comet_and_unsloth.ipynb

@@ -0,0 +1,357 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "6RF2EQaKDoxr"
+   },
+   "source": [
+    "<a href=\"https://www.comet.com/site/?utm_medium=colab&utm_source=comet-examples&utm_campaign=unsloth\" >\n",
+    "  <img src=\"https://cdn.comet.ml/img/notebook_logo.png\">\n",
+    "</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "KDjO5WbeDtp0"
+   },
+   "source": [
+    "# Comet and unsloth\n",
+    "\n",
+    "[Comet](https://www.comet.com/site/?utm_medium=colab&utm_source=comet-examples&utm_campaign=unsloth) is an MLOps platform designed to help data scientists and teams build better models faster! Comet provides tooling to track, explain, manage, and monitor your models in a single place! It works with Jupyter notebooks and scripts and-- most importantly--it's 100% free to get started!\n",
+    "\n",
+    "[unsloth](https://github.com/unslothai/unsloth) dramatically improves the speed and efficiency of LLM fine-tuning for models including Llama, Phi-3, Gemma, Mistral, and more. For a full listed of 100+ supported unsloth models, [see here](https://huggingface.co/unsloth).\n",
+    "\n",
+    "Instrument your torchtune training runs with Comet to start managing experiments with efficiency, reproducibility, and collaboration in mind.\n",
+    "\n",
+    "Find more information about [our integration with torchtune here](https://www.comet.com/docs/v2/integrations/third-party-tools/unsloth?utm_medium=colab&utm_source=comet-examples&utm_campaign=unsloth) or [learn about our other integrations here](https://www.comet.com/docs/v2/integrations?utm_medium=colab&utm_source=comet-examples&utm_campaign=unsloth)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "f0fhpJZgsYII"
+   },
+   "source": [
+    "## ⚙ Install and import dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "5zTSjY9r4cGc"
+   },
+   "outputs": [],
+   "source": [
+    "%pip install comet_ml \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\" \"torch>=2.4.0\" xformers trl peft accelerate bitsandbytes triton"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "p8GTqJOW88fI"
+   },
+   "outputs": [],
+   "source": [
+    "import comet_ml\n",
+    "\n",
+    "comet_ml.login()\n",
+    "exp = comet_ml.Experiment(project_name=\"comet-example-unsloth\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "vdOozTnqq8pL"
+   },
+   "outputs": [],
+   "source": [
+    "from huggingface_hub import notebook_login\n",
+    "\n",
+    "notebook_login()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "5bfxeJqKwpKj"
+   },
+   "source": [
+    "## ⚙ Download model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "bZei5Dvw7_kM"
+   },
+   "outputs": [],
+   "source": [
+    "from unsloth import FastLanguageModel\n",
+    "import torch\n",
+    "\n",
+    "max_seq_length = 2048\n",
+    "dtype = (\n",
+    "    None  # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\n",
+    ")\n",
+    "load_in_4bit = True"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "NG--Sbr1n5Cv"
+   },
+   "source": [
+    "Find the full list of [100+ supported unsloth models here](https://huggingface.co/unsloth). For a full list of supported 4-bit models see [here](https://huggingface.co/collections/unsloth/load-4bit-models-4x-faster-659042e3a41c3cbad582e734)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "L09m09Xs8Y2e"
+   },
+   "outputs": [],
+   "source": [
+    "model, tokenizer = FastLanguageModel.from_pretrained(\n",
+    "    model_name=\"unsloth/Meta-Llama-3.1-8B\",\n",
+    "    max_seq_length=max_seq_length,\n",
+    "    dtype=dtype,\n",
+    "    load_in_4bit=load_in_4bit,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "tbcH3PNSwRvy"
+   },
+   "source": [
+    "## ⚙ Add LoRA adapters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "BnXS0Trn8md9"
+   },
+   "outputs": [],
+   "source": [
+    "model = FastLanguageModel.get_peft_model(\n",
+    "    model,\n",
+    "    r=16,  # Suggested 8, 16, 32, 64, 128\n",
+    "    target_modules=[\n",
+    "        \"q_proj\",\n",
+    "        \"k_proj\",\n",
+    "        \"v_proj\",\n",
+    "        \"o_proj\",\n",
+    "        \"gate_proj\",\n",
+    "        \"up_proj\",\n",
+    "        \"down_proj\",\n",
+    "    ],\n",
+    "    lora_alpha=16,\n",
+    "    lora_dropout=0,  # Supports any, but = 0 is optimized\n",
+    "    bias=\"none\",  # Supports any, but = \"none\" is optimized\n",
+    "    use_gradient_checkpointing=\"unsloth\",  # True or \"unsloth\" for very long context\n",
+    "    random_state=3407,\n",
+    "    use_rslora=False,  # rank stabilized LoRA\n",
+    "    loftq_config=None,  # LoftQ\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "AoqQQ5n4wcfx"
+   },
+   "source": [
+    "## ⚙ Data preparation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "z9XFfoIz8sTI"
+   },
+   "outputs": [],
+   "source": [
+    "alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
+    "\n",
+    "### Instruction:\n",
+    "{}\n",
+    "\n",
+    "### Input:\n",
+    "{}\n",
+    "\n",
+    "### Response:\n",
+    "{}\"\"\"\n",
+    "\n",
+    "EOS_TOKEN = tokenizer.eos_token  # add EOS_TOKEN\n",
+    "\n",
+    "\n",
+    "def formatting_prompts_func(examples):\n",
+    "    instructions = examples[\"instruction\"]\n",
+    "    inputs = examples[\"input\"]\n",
+    "    outputs = examples[\"output\"]\n",
+    "    texts = []\n",
+    "    for instruction, input, output in zip(instructions, inputs, outputs):\n",
+    "        # add EOS_TOKEN, otherwise your generation will go on forever\n",
+    "        text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n",
+    "        texts.append(text)\n",
+    "    return {\n",
+    "        \"text\": texts,\n",
+    "    }\n",
+    "\n",
+    "\n",
+    "pass\n",
+    "\n",
+    "from datasets import load_dataset\n",
+    "\n",
+    "dataset = load_dataset(\"yahma/alpaca-cleaned\", split=\"train\")\n",
+    "dataset = dataset.map(\n",
+    "    formatting_prompts_func,\n",
+    "    batched=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "wljLKG7LwiHE"
+   },
+   "source": [
+    "## ⚙ Training"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "Ra62g_5f8vYo"
+   },
+   "outputs": [],
+   "source": [
+    "from trl import SFTTrainer\n",
+    "from transformers import TrainingArguments\n",
+    "from unsloth import is_bfloat16_supported\n",
+    "\n",
+    "trainer = SFTTrainer(\n",
+    "    model=model,\n",
+    "    tokenizer=tokenizer,\n",
+    "    train_dataset=dataset,\n",
+    "    dataset_text_field=\"text\",\n",
+    "    max_seq_length=max_seq_length,\n",
+    "    dataset_num_proc=2,\n",
+    "    packing=False,  # Can make training 5x faster for short sequences.\n",
+    "    args=TrainingArguments(\n",
+    "        per_device_train_batch_size=2,\n",
+    "        gradient_accumulation_steps=4,\n",
+    "        warmup_steps=5,\n",
+    "        # num_train_epochs = 1, # Set this for 1 full training run.\n",
+    "        max_steps=60,\n",
+    "        learning_rate=2e-4,\n",
+    "        fp16=not is_bfloat16_supported(),\n",
+    "        bf16=is_bfloat16_supported(),\n",
+    "        logging_steps=1,\n",
+    "        optim=\"adamw_8bit\",\n",
+    "        weight_decay=0.01,\n",
+    "        lr_scheduler_type=\"linear\",\n",
+    "        seed=3407,\n",
+    "        output_dir=\"outputs\",\n",
+    "    ),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "e5EHhd5zANxX"
+   },
+   "outputs": [],
+   "source": [
+    "trainer_stats = trainer.train()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "8II72I92Bm9s"
+   },
+   "outputs": [],
+   "source": [
+    "exp.end()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "0IdjUa_0pzPQ"
+   },
+   "source": [
+    "## ⚙ Inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "76su-yv9Aad0"
+   },
+   "outputs": [],
+   "source": [
+    "# alpaca_prompt = Copied from above\n",
+    "FastLanguageModel.for_inference(model)\n",
+    "inputs = tokenizer(\n",
+    "    [\n",
+    "        alpaca_prompt.format(\n",
+    "            \"Continue the fibonnaci sequence.\",  # instruction\n",
+    "            \"1, 1, 2, 3, 5, 8\",  # input\n",
+    "            \"\",  # output - leave this blank for generation\n",
+    "        )\n",
+    "    ],\n",
+    "    return_tensors=\"pt\",\n",
+    ").to(\"cuda\")\n",
+    "\n",
+    "outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)\n",
+    "tokenizer.batch_decode(outputs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "source": [],
+   "metadata": {
+    "id": "h6alQJYpwlAM"
+   },
+   "execution_count": null,
+   "outputs": []
+  }
+ ],
+ "metadata": {
+  "accelerator": "GPU",
+  "colab": {
+   "gpuType": "L4",
+   "machine_shape": "hm",
+   "provenance": [],
+   "toc_visible": true
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}