Tenosrs.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pytorch basics Tensors and gradients"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "importing pytorch library\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor\n",
    "\n",
    "As it is core pytorch is a library for processing tensors. A tensor is a number, vector, matrix, or any n-dimensional array."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "t1 is zero-dimensional tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(5.)\n",
      "torch.Size([])\n"
     ]
    }
   ],
   "source": [
    "t1 = torch.tensor(5 ,dtype=torch.float32)\n",
    "print(t1)\n",
    "print(t1.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "t2 is one-dimensional tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 2., 3., 4.])\n",
      "torch.Size([4])\n"
     ]
    }
   ],
   "source": [
    "t2 = torch.tensor([1.,2.,3.,4.])\n",
    "print(t2)\n",
    "print(t2.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "t3 is two-dimensional tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2., 3., 4.],\n",
      "        [5., 6., 7., 8.]])\n",
      "torch.Size([2, 4])\n"
     ]
    }
   ],
   "source": [
    "t3 = torch.tensor([[1.,2.,3.,4.],[5.,6.,7.,8.]])\n",
    "print(t3)\n",
    "print(t3.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "t4 is three-dimensional tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[ 1.,  2.,  3.],\n",
      "         [ 4.,  5.,  6.],\n",
      "         [ 7.,  8.,  9.]],\n",
      "\n",
      "        [[10., 11., 12.],\n",
      "         [13., 14., 15.],\n",
      "         [16., 17., 18.]]])\n",
      "torch.Size([2, 3, 3])\n"
     ]
    }
   ],
   "source": [
    "t4 = torch.tensor([[[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]],[[10.,11.,12.],[13.,14.,15.],[16.,17.,18.]]])\n",
    "print(t4)\n",
    "print(t4.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor operations and gradients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The value of x is  tensor(3.)\n",
      "The value of w is  tensor(4., requires_grad=True)\n",
      "The value of b is  tensor(5., requires_grad=True)\n",
      "The value of y is  tensor(17., grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# we can combine tensors with usual arthimatic operations\n",
    "x = torch.tensor(3.)\n",
    "w = torch.tensor(4.,requires_grad=True)\n",
    "b = torch.tensor(5.,requires_grad=True)\n",
    "\n",
    "y = w * x + b\n",
    "\n",
    "print(\"The value of x is \", x)\n",
    "print(\"The value of w is \", w)\n",
    "print(\"The value of b is \", b)\n",
    "print(\"The value of y is \", y)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compute derivative of y with respect to w, b as we made requires_grad = True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dy/dx None\n",
      "dy/dw tensor(3.)\n",
      "dy/db tensor(1.)\n"
     ]
    }
   ],
   "source": [
    "#  To compute gradient\n",
    "y.backward()\n",
    "\n",
    "# To display gradient\n",
    "\n",
    "print('dy/dx',x.grad)\n",
    "print('dy/dw',w.grad)\n",
    "print('dy/db',b.grad)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "creating a tensor with fixed value for each element"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[2, 2, 2, 2],\n",
      "        [2, 2, 2, 2],\n",
      "        [2, 2, 2, 2]])\n",
      "torch.Size([3, 4])\n"
     ]
    }
   ],
   "source": [
    "t5 = torch.full((3,4),2)\n",
    "print(t5)\n",
    "print(t5.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can concatinate two tensors but axis 1 dimensions must be same"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 4])\n",
      "torch.Size([3, 4])\n",
      "tensor([[1., 2., 3., 4.],\n",
      "        [5., 6., 7., 8.],\n",
      "        [2., 2., 2., 2.],\n",
      "        [2., 2., 2., 2.],\n",
      "        [2., 2., 2., 2.]])\n",
      "torch.Size([5, 4])\n"
     ]
    }
   ],
   "source": [
    "print(t3.shape)\n",
    "print(t5.shape)\n",
    "t6 = torch.cat((t3,t5))\n",
    "\n",
    "print(t6)\n",
    "print(t6.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can reshape tensor using reshape function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[1., 2., 3., 4., 5.],\n",
      "         [6., 7., 8., 2., 2.]],\n",
      "\n",
      "        [[2., 2., 2., 2., 2.],\n",
      "         [2., 2., 2., 2., 2.]]])\n",
      "torch.Size([2, 2, 5])\n"
     ]
    }
   ],
   "source": [
    "t7 = t6.reshape(2,2,5)\n",
    "print(t7)\n",
    "print(t7.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use special funtions also\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[ 0.8415,  0.9093,  0.1411, -0.7568, -0.9589],\n",
      "         [-0.2794,  0.6570,  0.9894,  0.9093,  0.9093]],\n",
      "\n",
      "        [[ 0.9093,  0.9093,  0.9093,  0.9093,  0.9093],\n",
      "         [ 0.9093,  0.9093,  0.9093,  0.9093,  0.9093]]])\n"
     ]
    }
   ],
   "source": [
    "t8 = torch.sin(t7)\n",
    "\n",
    "print(t8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.]])\n",
      "torch.Size([2, 2])\n"
     ]
    }
   ],
   "source": [
    "t9 = torch.ones((2,2))\n",
    "\n",
    "print(t9)\n",
    "print(t9.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpoearablity with Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 2. 3.]\n",
      " [4. 5. 6.]]\n",
      "(2, 3)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "n1 = np.array([[1,2,3.],[4,5,6]])\n",
    "\n",
    "print(n1)\n",
    "print(n1.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can convert numpy array to tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2., 3.],\n",
      "        [4., 5., 6.]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "t11 = torch.from_numpy(n1)\n",
    "\n",
    "print(t11)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(dtype('float64'), torch.float64)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n1.dtype, t11.dtype"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can convert tensors to numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 2. 3.]\n",
      " [4. 5. 6.]]\n"
     ]
    }
   ],
   "source": [
    "n2 = t11.numpy()\n",
    "\n",
    "print(n2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why we need a library like PyTorch since Numpy already provides data structure and utilities for working with multi-dimensional numeric data.\n",
    "\n",
    "There are two main resaons.\n",
    "\n",
    "\n",
    "1. $\\textbf{Autograd:}$ The ability to automatically compute gradients for tensor operations is essential for training deep learning models.\n",
    "\n",
    "2. $\\textbf{GPU Support:}$ While working with massive datasets and large models, PyTorch tensor operations can be performed efficiently using a Graphical Processing Unit (GPU). Computations that might typically take hours can be completed within minutes using GPUs.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}