t81_559_class_11_1_finetune.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kxu1Gfhx1pHg"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_11_1_finetune.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZbNbAV281pHh"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 11: Finetuning**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HwnvSYEQ1pHi"
   },
   "source": [
    "# Module 11 Material\n",
    "\n",
    "Module 11: Finetuning\n",
    "\n",
    "* **Part 11.1: Understanding Finetuning** [[Video]](https://www.youtube.com/watch?v=fflySydZABM) [[Notebook]](t81_559_class_11_1_finetune.ipynb)\n",
    "* Part 11.2: Finetuning from the Dashboard [[Video]](https://www.youtube.com/watch?v=RIJj1QLk-V4) [[Notebook]](t81_559_class_11_2_dashboard.ipynb)\n",
    "* Part 11.3: Finetuning from Code [[Video]](https://www.youtube.com/watch?v=29tUrxrneOs) [[Notebook]](t81_559_class_11_3_code.ipynb)\n",
    "* Part 11.4: Evaluating your Model [[Video]](https://www.youtube.com/watch?v=MrwFSG4PWUY) [[Notebook]](t81_559_class_11_4_eval.ipynb)\n",
    "* Part 11.5: Finetuning for Text to Image [[Video]](https://www.youtube.com/watch?v=G_FYFSzkB5Y&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_559_class_11_5_image.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v2MPPX0c1pHi"
   },
   "source": [
    "# Part 11.1: Understanding Finetuning\n",
    "\n",
    "OpenAI offers fine-tuning capabilities that allow users to customize models like GPT-4 for specific use cases by training them on domain-specific data. Fine-tuning involves providing a curated dataset to adjust the model's weights, which improves performance on specialized tasks such as answering questions, generating more accurate predictions, or adhering to brand guidelines. OpenAI’s fine-tuning process supports prompt customization, enabling the model to align better with user expectations and handle specific instructions more effectively. This approach is particularly useful for industries like finance, healthcare, or customer service, where domain expertise and precision are critical.\n",
    "\n",
    "The complete documentation from OpenAI that covers finetuning can be found here:\n",
    "\n",
    "* [OpenAI Finetuning Guide](https://platform.openai.com/docs/guides/fine-tuning)\n",
    "\n",
    "\n",
    "Fine-tuning is a powerful technique that allows you to optimize pre-trained models, tailoring them to specific tasks and achieving higher performance than standard prompting. While models available through APIs like OpenAI’s are pre-trained on vast datasets, fine-tuning enhances their abilities, resulting in several key benefits:\n",
    "\n",
    "* Higher quality results than can be achieved by simple prompting\n",
    "* Ability to train on more examples than can fit in a single prompt\n",
    "* Token savings through shorter, more efficient prompts\n",
    "* Lower latency when making requests, as the model becomes more specialized\n",
    "\n",
    "Common usecases for fine tuning include:\n",
    "\n",
    "* **Style and Tone** - If you want to change the overall tone that the LLM responds with.\n",
    "* **Structured Output** - If you want to force the output to always be JSON, XML, or another structure.\n",
    "* **Tool Calling** - If you want to enable the LLM to make use of tools in a specific way.\n",
    "* **Function Calling** - If you want to enable the LLM to call functions in a specific way.\n",
    "\n",
    "### What is Fine-Tuning?\n",
    "OpenAI’s text generation models are pre-trained on a broad corpus of data, enabling them to handle a wide range of tasks. However, out-of-the-box usage often requires users to guide the model with carefully designed prompts and examples. This technique, known as \"few-shot learning,\" can be effective but is limited by the number of examples that can be provided in a prompt.\n",
    "\n",
    "Fine-tuning takes this a step further by allowing the model to be trained on a much larger set of examples. This process updates the model’s internal weights to better handle the specific tasks or domains you're focused on, ultimately producing more accurate and reliable results. After fine-tuning, the model needs fewer examples to perform at a high level, which reduces both prompt length and cost while improving response times.\n",
    "\n",
    "### The Fine-Tuning Process\n",
    "Fine-tuning typically involves the following steps:\n",
    "\n",
    "* Prepare and upload training data\n",
    "    Your training data should include a comprehensive set of examples that represent the task or domain you want to improve.\n",
    "\n",
    "* Train a new fine-tuned model\n",
    "    The model is retrained using your custom dataset, adjusting its parameters to better suit your needs.\n",
    "\n",
    "* Evaluate results and iterate\n",
    "    After training, you’ll need to assess the model's performance. If necessary, you can refine your data or adjust the model's configuration and retrain for further improvements.\n",
    "\n",
    "### Deploy and use your fine-tuned model\n",
    "Once satisfied with the model’s performance, it can be deployed for use in production, providing faster and more cost-efficient responses tailored to your specific tasks.\n",
    "\n",
    "Fine-tuning enables more efficient and specialized models, allowing you to save time and resources while achieving superior results. For more information about the cost structure and pricing for fine-tuned models, visit our pricing page.\n",
    "\n",
    "### Data Formats\n",
    "\n",
    "\n",
    "The format for LLM training is a conversational style used for training a dialogue-based model, such as in a chatbot. This format organizes exchanges into a sequence of \"messages,\" where each message is assigned a specific \"role\" (either \"system,\" \"user,\" or \"assistant\") and \"content\" that captures the text provided by the user or generated by the assistant. It allows the model to learn how to respond in context by reviewing a series of interactions. The \"system\" message typically sets the tone or instructions for the assistant (such as sarcasm), while subsequent messages simulate a conversation between the user and the assistant, helping the model to understand how to craft contextually appropriate and stylistically consistent responses.\n",
    "\n",
    "```\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"Marv is a factual chatbot that is also sarcastic.\"}, {\"role\": \"user\", \"content\": \"What's the capital of France?\"}, {\"role\": \"assistant\", \"content\": \"Paris, as if everyone doesn't know that already.\"}]}\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"Marv is a factual chatbot that is also sarcastic.\"}, {\"role\": \"user\", \"content\": \"Who wrote 'Romeo and Juliet'?\"}, {\"role\": \"assistant\", \"content\": \"Oh, just some guy named William Shakespeare. Ever heard of him?\"}]}\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"Marv is a factual chatbot that is also sarcastic.\"}, {\"role\": \"user\", \"content\": \"How far is the Moon from Earth?\"}, {\"role\": \"assistant\", \"content\": \"Around 384,400 kilometers. Give or take a few, like that really matters.\"}]}\n",
    "```\n",
    "\n",
    "\n",
    "### Understanding Finetuning Costs\n",
    "\n",
    "\n",
    "Fine-tuning a large language model (LLM) can be costly, not only in terms of the initial training process but also due to the ongoing expenses of serving and querying the model. Training an LLM requires significant computational resources, especially when working with vast datasets to adjust the model's parameters, which can lead to high costs for cloud services or specialized hardware. In addition, once fine-tuned, hosting the model for production use involves maintaining expensive infrastructure to handle real-time queries, which can scale based on usage and require optimization for cost-effective deployment. The combined training and operational costs can quickly accumulate, making fine-tuning a resource-intensive process.\n",
    "\n",
    "The costs for the training and inference stages have changed over time, and continue to evolve. You can find the current prices here:\n",
    "\n",
    "* [OpenAI Pricing](https://openai.com/api/pricing/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}