11_hw_cp220_Q.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/AshrithSagar/CP220-M4RAS-2024/blob/main/11_hw_cp220_Q.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SHF4Zs1fDZZA"
   },
   "source": [
    "---\n",
    "**HW11**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fFc3JCl0DlmO"
   },
   "source": [
    "With a continuous rv, we can associate a non-negative function $f_X$ which is called the probability density function (pdf), such that:\n",
    "\n",
    "$P(X \\in B) = \\int_B f_X(x)dx$\n",
    "\n",
    "where $B \\subset ℝ$\n",
    "\n",
    "and $∫_{-∞}^{∞} f_X(x)dx = 1$\n",
    "\n",
    "Note that the probility for a small interval $[x, x+\\delta]$ is $P(X \\in [x, x+\\delta]) = \\int_x^{x+\\delta} f_X(x)dx \\approx f_X(x)\\delta$\n",
    "\n",
    "Thus probability for any particular value of X is 0.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Pcaq3VpHv6Ik"
   },
   "source": [
    "*$\\underline{Uniform\\ rv}$*\n",
    "\n",
    "$X\\sim\\ Unif(a,b)$, where $a < b$\n",
    "is when\n",
    "\n",
    "$f_X(x) = 1/(b-a)$ for $a \\leq x \\leq b$\n",
    "\n",
    " and 0 else where.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V3QFieBNwZGb"
   },
   "source": [
    "*$\\underline{Expectation, Moments\\ and\\ Variance}$*\n",
    "\n",
    "\n",
    "$E g(X) = \\int_{-∞}^∞ g(x)f_X(x) dx$\n",
    "\n",
    "The mean is EX\n",
    "\n",
    "and Variance is $\\sigma_X^2 = EX^2 - (EX)^2$\n",
    "\n",
    "And the $k^{th}$ raw moment is $EX^k = \\int_{-∞}^∞ X^k f_X(x) dx$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W0zDGrZoxtKT"
   },
   "source": [
    "*Example 1*:\n",
    "\n",
    "Let $X\\ \\tilde\\ Unif(a,b)$\n",
    "\n",
    "Then $EX = \\int_a^b x/(b-a)dx = \\frac{b^2-a^2}{2(b-a)} = \\frac{b+a}{2}$\n",
    "\n",
    "$EX^2 = \\int_a^b x^2/(b-a)dx = \\frac{b^3-a^3}{3(b-a)} = \\frac{b^2+ab+a^2}{3}$\n",
    "\n",
    "$Var(X) = \\sigma_X^2 = \\frac{b^2+ab+a^2}{3} - \\frac{b^2+2ab+a^2}{4} = \\frac{(b-a)^2}{12}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xVajxDiQzRgs"
   },
   "source": [
    "*$\\underline{Exponential\\ rv}$*\n",
    "\n",
    "$X\\sim Exp(\\lambda)$ is exponential then:\n",
    "\n",
    "$f_X(x) = \\lambda e^{-\\lambda x}$ for $x \\geq 0$\n",
    "\n",
    "and 0 otherwise.\n",
    "\n",
    "$EX = \\int_0^{∞} x\\lambda e^{-\\lambda x} dx = \\frac{1}{\\lambda}$\n",
    "\n",
    "$Var(X) = \\frac{1}{\\lambda^2}$\n",
    "\n",
    "$P(X\\geq a) = \\int_a^∞ \\lambda e^{-\\lambda x} dx = e^{-\\lambda a}$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QgzD3Qik1U8y"
   },
   "source": [
    "**Cumulative Distribution Function**\n",
    "\n",
    "$F_X(x) = P(X \\leq x) = \\int_{-∞}^x f_X(x)dx$\n",
    "\n",
    "is called the cumulative distribution function.\n",
    "\n",
    "*Conversely*,\n",
    "\n",
    "$f_X(x) = \\frac{d}{dx} F_X(x)$\n",
    "\n",
    "\n",
    "\n",
    "For discrete rv, $F_X(x) = \\sum_{k \\leq x} p_X(k)$\n",
    "\n",
    "and $p_X(x) = F_X(x) - F_X(x-1)$\n",
    "\n",
    "\n",
    "*Example:*\n",
    "\n",
    "The geometric rv, $Y\\ \\tilde\\ Geom(p)$ is connected to $X\\sim\\ Exp(\\lambda)$ as follows;\n",
    "\n",
    "$F_Y(n) = \\sum_{k=1}^n p(1-p)^k = 1 - (1-p)^n$\n",
    "\n",
    "$F_X(x) = \\int_0^x \\lambda e^{-\\lambda x} dx = 1 - e^{-\\lambda x}$\n",
    "\n",
    "By discretising with steps of $\\delta$, and putting $x = n \\delta$, and using $p = 1 - e^{-\\lambda \\delta}$, we find that:\n",
    "\n",
    "$F_Y(n) = F_X(n\\delta)$ for $n=1,2, ... $\n",
    "\n",
    "Fig 3.8 (Bertsekas):\n",
    "\n",
    "<img src='https://drive.google.com/uc?export=view&id=17K_NGZbrSGazPrad11v8bYUpd54HJY41' width=400px>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eL7X1YJe394i"
   },
   "source": [
    "*$\\underline{Normal\\ or\\ Gaussian\\ pdf}$*\n",
    "\n",
    "$X\\sim\\ N(\\mu,\\sigma)$\n",
    "\n",
    "then,\n",
    "\n",
    "$f_X(x) = \\frac{1}{\\sqrt{2\\pi}\\sigma} e^{\\frac{-(x-\\mu)^2}{\\sigma^2}}$\n",
    "\n",
    "$EX = \\mu$\n",
    "\n",
    "$Var(X) = \\sigma^2$\n",
    "\n",
    "Fig 3.9 Bertsekas:\n",
    "\n",
    "\n",
    "<img src='https://drive.google.com/uc?export=view&id=17M6_00Pb1rmzVBdHmshjRuHWAQqLIxqb' width=400px>\n",
    "\n",
    "*$\\underline{Standard\\ Normal}$*\n",
    "\n",
    "Normal distribution with $\\mu=0$ and $\\sigma=1$ is called a standard normal $N(0,1)$ with pdf $\\frac{1}{\\sqrt{2\\pi}}e^{\\frac{-x^2}{2}}$.\n",
    "\n",
    "CDF of standard normal:\n",
    "\n",
    "$\\Phi(x) = P(X \\leq x) = \\int_{-\\infty}^x \\frac{1}{\\sqrt{2\\pi}}e^{\\frac{-y^2}{2}}dy$\n",
    "\n",
    "There is no analytical formula - but this is calculated numerically. For hand estimation, one can use standard normal tables.\n",
    "\n",
    "Let $Z\\sim\\ N(\\mu, \\sigma)$, then $P(Z \\leq z) = \\int_{-\\infty}^z \\frac{1}{\\sqrt{2\\pi}\\sigma}e^{\\frac{-(y-\\mu)^2}{2\\sigma^2}}dy$\n",
    "\n",
    "By change of variable $u = \\frac{y-\\mu}{\\sigma}$, we have:\n",
    "\n",
    "$P(Z \\leq z) = P(X \\leq \\frac{z-\\mu}{\\sigma}) = \\int_{-\\infty}^{\\frac{z-\\mu}{\\sigma}} \\frac{1}{\\sqrt{2\\pi}}e^{\\frac{-u^2}{2}}du = \\Phi(\\frac{z-\\mu}{\\sigma})$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H8fmRbFQNW9M"
   },
   "source": [
    "*$\\underline{Linear\\ function\\ of\\ Gaussian\\ rv\\ is\\ a\\ Gaussian}$*\n",
    "\n",
    "Let $X\\ \\sim\\ N(\\mu,\\sigma)$\n",
    "\n",
    "If $Y = aX + b$\n",
    "\n",
    "$Y\\sim\\ N(a\\mu+b,a\\sigma)$\n",
    "\n",
    "$EY = a\\mu + b$\n",
    "\n",
    "$EY^2 = E(a^2X^2 + 2aXb + b^2) = a^2EX^2 + 2aEXb + b^2$\n",
    "\n",
    "$Var(Y) = EY^2 - (EY)^2 = a^2 \\sigma^2$\n",
    "\n",
    "$F_Y(y) = P(Y \\leq y) = P(X \\leq \\frac{y-b}{a}) = F_X(\\frac{y-b}{a})$\n",
    "\n",
    "$f_Y(y) = \\frac{dF_Y(y)}{dy} = \\frac{d}{dy}\\int_{-\\infty}^{\\frac{y-b}{a}} f_X(x)dx = \\frac{1}{a}f_X(\\frac{y-b}{a}) = N(a\\mu,a\\sigma) $\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Tjcc9oHSjgFM"
   },
   "source": [
    "---\n",
    "**Joint, Marginal, Independence, Conditioning**\n",
    "\n",
    "*$\\underline{Joint}$*\n",
    "\n",
    "$f_{XY}(x,y)$ is a joint PDF of two rvs, $X,Y$ such that:\n",
    "\n",
    "a) $f_{XY}(x,y) \\geq 0 $\n",
    "\n",
    "b) $P((X,Y) \\in B) = \\int\\int_B f_{XY}(x,y)dxdy$\n",
    "\n",
    "\n",
    "For example, to find the probability of $X \\in (a,b)$ and $Y \\in (c,d)$,\n",
    "\n",
    "$P(a\\leq X \\leq b, c \\leq Y \\leq d) = \\int_a^b\\int_c^d f_{XY}(x,y)dxdy$\n",
    "\n",
    "*$\\underline{Marginal}$*\n",
    "\n",
    "Given a joint distribution, the marginal pdf of a subset of variables is obtained by integrating out the remaning across their entire domain.\n",
    "\n",
    "for example, given $f_{XYZW}(x,y,z,w)$\n",
    "\n",
    "$f_{XY}(x,y) = \\int_z\\int_w f_{XYZW}(x,y,z,w) dzdw$\n",
    "\n",
    "*$\\underline{Joint\\ CDF}$*\n",
    "\n",
    "$F_{XY}(x,y) = P(X \\leq x, Y \\leq y)$\n",
    "\n",
    "\n",
    "and conversely,\n",
    "\n",
    "$f_{XY}(x,y) = \\frac{\\delta^2 F_{XY}}{\\delta x \\delta y}$\n",
    "\n",
    "\n",
    "*$\\underline{Conditional\\ pdf}$*\n",
    "\n",
    "Given an event C, the conditional distribution is such that:\n",
    "\n",
    "$P(X \\in B | C) = \\int_B f_{X|C}(x)dx$\n",
    "\n",
    "\n",
    "Specifically, for joint continuous rv, we can also define:\n",
    "\n",
    "$f_{X|Y=y}(x) = \\frac{f_{XY}(x,y)}{f_Y(y)}$\n",
    "\n",
    "So,\n",
    "\n",
    "$f_{XY}(x,y) = f_{X|Y=y}(x)f_Y(y)$\n",
    "\n",
    "\n",
    "*$\\underline{Independence}$*\n",
    "\n",
    "X,Y are independent if\n",
    "\n",
    "$f_{XY}(x,y) = f_X(x)f_Y(y)$\n",
    "\n",
    "\n",
    "*$\\underline{Expectation}$*\n",
    "\n",
    "$E_{XY}(g(X,Y)) = \\int_x\\int_y g(x,y)f_{XY}(x,y) dxdy$\n",
    "\n",
    "and\n",
    "\n",
    "$E_{XY}(aX+bY) = aE_XX + bE_YY$\n",
    "\n",
    "\n",
    "\n",
    "$Var(X,Y) = Var(X) + Var(Y)$ if X, Y are independent.\n",
    "\n",
    "\n",
    "*$\\underline{Conditional\\ Expectation}$*\n",
    "\n",
    "\n",
    "\n",
    "$E[X|Y=y] = \\int_x xf_{X|Y=y}(x) dx$\n",
    "\n",
    "*Total Expectation Theorem*\n",
    "\n",
    "$EX = E_Y E_{X|Y=y} X = \\int_y E_{X|Y=y} f_Y(y) dy$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Da6sVg5gtt8l"
   },
   "source": [
    "---\n",
    "*Mixed discrete/continuous*\n",
    "\n",
    "Let $X$ indicate amount of relative humidity and takes value from 0 to 100.\n",
    "\n",
    "Let $Y$ be the rv takes value 1 if it rains and 0 otherwise.\n",
    "\n",
    "Then let\n",
    "\n",
    "$f_{X|Y=1}(x) = 1/20$ for $70 \\leq X \\leq 90$ and 0 elsewhere\n",
    "\n",
    "$f_{X|Y=0}(x) = 1/30$ for $45 \\leq X \\leq 75$ and 0 elsewhere\n",
    "\n",
    "Let $P(Y=1) = 0.6$ and $P(Y=0) = 0.4$\n",
    "\n",
    "\n",
    "Then we have the joint probability:\n",
    "\n",
    "\n",
    "$P(a \\leq X \\leq b, \"it\\ rains\") = \\int_a^b f_{X|Y=1}(x)P(Y=1)$\n",
    "\n",
    "$P(a \\leq X \\leq b, \"it\\ doesnt\\ rain\") = \\int_a^b f_{X|Y=0}(x)P(Y=0)$\n",
    "\n",
    "Average humidity is:\n",
    "\n",
    "\n",
    "$EX = E_Y E_{X|Y=y} X = \\sum_y E_{X|Y=y} p_Y(y) = 0.6*80+0.4*60=72$\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o0r4riIYvsjl"
   },
   "source": [
    "---\n",
    "*Bayes Rule*\n",
    "\n",
    "\n",
    "$f_{XY}(x,y) = f_{X|Y=y}(x)f_Y(y) = f_{Y|X=x}(y)f_X(x)$\n",
    "\n",
    "and hence\n",
    "\n",
    "$f_{X|Y=y} = \\frac{f_{Y|X=x}(y)f_X(x)}{f_Y(y)}$\n",
    "\n",
    "\n",
    "*Example*\n",
    "\n",
    "a) For previous example, given that the humidity is 73, what is the chance that it has rained?\n",
    "\n",
    "$P(Y=1|x=73) = \\frac{f_{X|Y=1}(73)P(Y=1)}{f_X(73)} = \\frac{f_{X|Y=1}(73)P(Y=1)}{f_{X|Y=1}(73)P(Y=1)+f_{X|Y=0}(73)P(Y=0)} = \\frac{1/20*0.6}{1/20*0.6+1/30*0.4} = 0.69$\n",
    "\n",
    "b) if humidity is 48, what is the chance it has rained?\n",
    "\n",
    "We can guess looking at the conditional pdf that itwouldnt have rained.\n",
    "\n",
    "$P(Y=1|x=48) = \\frac{f_{X|Y=1}(48)P(Y=1)}{f_X(48)} = \\frac{f_{X|Y=1}(48)P(Y=1)}{f_{X|Y=1}(48)P(Y=1)+f_{X|Y=0}(48)P(Y=0)} = \\frac{0*0.6}{0*0.6+1/30*0.4} = 0$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EeS9DpH2PMVL"
   },
   "source": [
    "---\n",
    "**Q1**\n",
    "\n",
    "Problem 1. Let X be uniformly distributed in the unit interval [0, 1] . Consider the\n",
    "random variable $Y = g(X)$, where\n",
    "\n",
    "$g(x) = 1$ if $x \\leq 1/3$ and\n",
    "\n",
    "$g(x) = 0 $ if $x \\gt 1/3$\n",
    "\n",
    "Find the expected value of Y by first deriving its PMF. Verify the result using the expected value rule."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "azmuhojNPirM"
   },
   "source": [
    "---\n",
    "Ans:\n",
    "\n",
    "We're given that $X\\sim U[0, 1]$ and\n",
    "$\n",
    "\\displaystyle\n",
    "g(x) = \n",
    "\\begin{cases}\n",
    "1 & \\text{if } x \\leq 1/3\\\\\n",
    "0 & \\text{if } x > 1/3\n",
    "\\end{cases}\n",
    "$\n",
    "\n",
    "We know that\n",
    "$\n",
    "\\displaystyle\n",
    "f_X (x) =\n",
    "\\begin{cases}\n",
    "1 & \\text{if } 0 \\leq x \\leq 1\\\\\n",
    "0 & \\text{otherwise}\n",
    "\\end{cases}\n",
    "$\n",
    "and\n",
    "$\n",
    "\\displaystyle\n",
    "F_X (x) =\n",
    "\\begin{cases}\n",
    "0 & \\text{if } x < 0\\\\\n",
    "x & \\text{if } 0 \\leq x \\leq 1\\\\\n",
    "1 & \\text{if } x > 1\n",
    "\\end{cases}\n",
    "$\n",
    "\n",
    "Then the distribution function of $Y = g(X)$ is given by\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "F_Y (y) = P[Y \\leq y] = P[g(X) \\leq y] = P[X \\leq g^{-1} (y)] = F_X (g^{-1} (y))\n",
    "= \\begin{cases}\n",
    "0 & \\text{if } g^{-1} (y) < 0\\\\\n",
    "x & \\text{if } 0 \\leq g^{-1} (y) \\leq 1\\\\\n",
    "1 & \\text{if } g^{-1} (y) > 1\n",
    "\\end{cases}\n",
    "$\n",
    "\n",
    "We can see that $Y$ is a binary random variable, and we have\n",
    "$\n",
    "\\displaystyle\n",
    "g^{-1} (y) =\n",
    "\\begin{cases}\n",
    "(-\\infty, 1/3] & \\text{if } y = 1\\\\\n",
    "(1/3, 1] & \\text{if } y = 0\n",
    "\\end{cases}\n",
    "$\n",
    "\n",
    "Thus, the PMF of $Y$ is given by\n",
    "$\n",
    "\\displaystyle\n",
    "f_Y (y) = F_Y (y) - F_Y (y - 1) = F_X (g^{-1} (y)) - F_X (g^{-1} (y - 1))\n",
    "$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4a52xeqmQpcS"
   },
   "source": [
    "---\n",
    "**Q2**\n",
    "\n",
    "Problem 2. **Laplace RV**\n",
    "\n",
    "$f_X(x) = \\frac{\\lambda}{2}e^{-\\lambda|x|}$\n",
    "\n",
    "where $\\lambda$ is a positive scalar. Verify that $f_X(x)$ satisfies the normalization condition, and\n",
    "evaluate the mean and variance of X."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k17LtKCkRByg"
   },
   "source": [
    "---\n",
    "Ans:\n",
    "\n",
    "We're given that\n",
    "$\n",
    "\\displaystyle\n",
    "f_X (x) = \\frac{\\lambda}{2} e^{-\\lambda |x|}\n",
    "$\n",
    "for some positive scalar $\\lambda$.\n",
    "\n",
    "The normalization condition states that\n",
    "$\n",
    "\\displaystyle\n",
    "\\int_{-\\infty}^{\\infty} f_X (x) dx = 1\n",
    "$\n",
    "\n",
    "We have\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "\\implies\n",
    "\\int_{-\\infty}^{\\infty} f_X (x) dx\n",
    "= \\int_{-\\infty}^{\\infty} \\frac{\\lambda}{2} e^{-\\lambda |x|} dx\n",
    "= \\int_{-\\infty}^{0} \\frac{\\lambda}{2} e^{\\lambda x} dx + \\int_{0}^{\\infty} \\frac{\\lambda}{2} e^{-\\lambda x} dx\n",
    "= 2 \\frac{\\lambda}{2} \\int_{0}^{\\infty} e^{-\\lambda x} dx\n",
    "= \\lambda \\left[ -\\frac{1}{\\lambda} e^{-\\lambda x} \\right]_{0}^{\\infty}\n",
    "= \\left[ -e^{-\\lambda x} \\right]_{0}^{\\infty}\n",
    "= 0 - (-1)\n",
    "= 1\n",
    "$\n",
    "\n",
    "Hence, $f_X (x)$ satisfies the normalization condition.\n",
    "\n",
    "The mean can be computed as\n",
    "\n",
    "$\n",
    "\\begin{aligned}\n",
    "E[X]\n",
    "&\n",
    "= \\int_{-\\infty}^{\\infty} x f_X (x) dx\n",
    "= \\int_{-\\infty}^{\\infty} x \\frac{\\lambda}{2} e^{-\\lambda |x|} dx\n",
    "= \\int_{-\\infty}^{0} x \\frac{\\lambda}{2} e^{\\lambda x} dx\n",
    "    + \\int_{0}^{\\infty} x \\frac{\\lambda}{2} e^{-\\lambda x} dx\n",
    "\\\\\n",
    "\\implies\n",
    "\\int_{-\\infty}^{0} x \\frac{\\lambda}{2} e^{\\lambda x} dx\n",
    "&\n",
    "= \\frac{\\lambda}{2} \\left[ x \\frac{1}{\\lambda} e^{\\lambda x} - \\int \\frac{1}{\\lambda} e^{\\lambda x} dx \\right]_{-\\infty}^{0}\n",
    "= \\frac{\\lambda}{2} \\left[ - \\frac{1}{\\lambda^2} e^{\\lambda x} \\right]_{-\\infty}^{0}\n",
    "= -\\frac{1}{2\\lambda}\n",
    "\\\\\n",
    "\\int_{0}^{\\infty} x \\frac{\\lambda}{2} e^{-\\lambda x} dx\n",
    "&\n",
    "= \\frac{\\lambda}{2} \\left[ x \\frac{-1}{\\lambda} e^{-\\lambda x} - \\int \\frac{-1}{\\lambda} e^{-\\lambda x} dx \\right]_{0}^{\\infty}\n",
    "= \\frac{1}{2} \\int_{0}^{\\infty} e^{-\\lambda x} dx\n",
    "= \\frac{1}{2} \\left[ -\\frac{1}{\\lambda} e^{-\\lambda x} \\right]_{0}^{\\infty}\n",
    "= \\frac{1}{2\\lambda}\n",
    "\\\\\n",
    "\\implies\n",
    "E[X]\n",
    "&\n",
    "= -\\frac{1}{2\\lambda} + \\frac{1}{2\\lambda}\n",
    "= 0\n",
    "\\end{aligned}\n",
    "$\n",
    "\n",
    "The variance can be computed as\n",
    "\n",
    "$\n",
    "\\begin{aligned}\n",
    "Var[X]\n",
    "&\n",
    "= E[X^2] - (E[X])^2\n",
    "= \\int_{-\\infty}^{\\infty} x^2 f_X (x) dx - (E[X])^2\n",
    "\\end{aligned}\n",
    "$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "62wZVlmOVBf7"
   },
   "source": [
    "---\n",
    "**Q3**\n",
    "\n",
    "Problem 5. Consider a triangle and a point chosen within the triangle according to the uniform probability law. Let X be the distance from the point to the base of the triangle. Given the height of the triangle, find the CDF and the PDF of X."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8XkEMZ1vVE5B"
   },
   "source": [
    "Ans:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yqCvB-JOW_Kg"
   },
   "source": [
    "**Q4**\n",
    "\n",
    "Problem 6. Calamity Jane goes to the bank to make a withdrawal, and is equally likely to find 0 or 1 customers ahead of her. The service time of the customer ahead, if present, is exponentially distributed with parameter $\\lambda$. What is the CDF of Jane's waiting time?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U6_8f179XKMq"
   },
   "source": [
    "Ans:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wJL9WxRlYiei"
   },
   "source": [
    "**Q5**\n",
    "\n",
    "\n",
    "Problem 7. Alvin throws darts at a circular target of radius r and is equally likely to hit any point in the target. Let X be the distance of Alvin's hit from the center.\n",
    "\n",
    "(a) Find the PDF, the mean. and the variance of X.\n",
    "\n",
    "(b) The target has an inner circle of radius t.If $X \\leq t$,Alvin gets a\n",
    "scoreof S= 1/X. Otherwise his score is S = O. Find the CDF of S. Is S a continuous random variable?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8PD7LOl_ZJlU"
   },
   "source": [
    "Ans:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_71rpPQZGl_J"
   },
   "source": [
    "---\n",
    "**Q6**\n",
    "\n",
    "Problem 8. Consider two continuous random variables, Y, Z and a random\n",
    "variable X that is equal to Y with probability p and to Z with probability 1-p\n",
    "\n",
    "(a) Show that the PDF of X is given by\n",
    "\n",
    "$f_X(x) = pf_Y(x) + (1 - p)f_Z(x)$\n",
    "\n",
    "\n",
    "(b) Calculate the CDF of the two-sided exponential random variable that has PDF given by\n",
    "\n",
    "$f_X(x) = p\\lambda e^{\\lambda x}$ if $x <0$\n",
    "\n",
    "$\\ \\ \\ = (1-p)\\lambda e^{-\\lambda x}$ if $x \\geq 0$\n",
    "\n",
    "where $\\lambda > 0$ and $0 < p < 1$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QurKNWhNHenB"
   },
   "source": [
    "Ans:\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "58xFLBZKLA0j"
   },
   "source": [
    "---\n",
    "**Q7**\n",
    "\n",
    "Problem 11 . Let X and Y be normal random variables with means 0 and 1 , respec­tively, and variances 1 and 4, respectively.\n",
    "\n",
    "(a) Find $P(X \\leq 1.5)$ and $P(X \\leq -1)$\n",
    "\n",
    "(b) Find the PDF of (Y - 1)/2.\n",
    "\n",
    "(c) Find $P(-1 \\leq Y \\leq 1)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lhUQp7tGLwdj"
   },
   "source": [
    "Ans:\n",
    "\n",
    "We have $X \\sim \\mathcal{N}(0, 1)$ and $Y \\sim \\mathcal{N}(1, 4)$.\n",
    "\n",
    "$\n",
    "\\begin{aligned}\n",
    "\\implies\n",
    "f_X (x)\n",
    "& = \\frac{1}{\\sqrt{2\\pi}} e^{-\\frac{x^2}{2}}\n",
    ", \\qquad\n",
    "f_Y (y)\n",
    "& = \\frac{1}{2\\sqrt{2\\pi}} e^{-\\frac{(y-1)^2}{8}}\n",
    "\\end{aligned}\n",
    "$\n",
    "\n",
    "(a)\n",
    "\n",
    "$\n",
    "\\implies\n",
    "\\displaystyle\n",
    "P(X \\leq 1.5)\n",
    "= \\int_{-\\infty}^{1.5} f_X (x) dx\n",
    "= \\int_{-\\infty}^{1.5} \\frac{1}{\\sqrt{2\\pi}} e^{-\\frac{x^2}{2}} dx\n",
    "$\n",
    "\n",
    "$\n",
    "\\implies\n",
    "P(X \\leq -1)\n",
    "= \\int_{-\\infty}^{-1} f_X (x) dx\n",
    "= \\int_{-\\infty}^{-1} \\frac{1}{\\sqrt{2\\pi}} e^{-\\frac{x^2}{2}} dx\n",
    "$\n",
    "\n",
    "(b)\n",
    "With $Z = (Y - 1)/2$, we have\n",
    "\n",
    "$\n",
    "\\begin{aligned}\n",
    "F_Z (z)\n",
    "&\n",
    "= P[Z \\leq z]\n",
    "= P\\left[\\frac{Y - 1}{2} \\leq z\\right]\n",
    "= P[Y \\leq 2z + 1]\n",
    "\\end{aligned}\n",
    "$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "h_Ngx-PcOB-M"
   },
   "source": [
    "---\n",
    "**Q8**\n",
    "\n",
    "Problem 15. A point is chosen at random (according to a uniform PDF) within a semicircle of the form $\\{(x, y)| x^2 + y^2 \\leq r^2, y \\geq 0\\}$, for some given $r > 0$\n",
    "\n",
    "(a) Find the joint PDF of the coordinates X and Y of the chosen point.\n",
    "\n",
    "(b) Find the marginal PDF of Y and use it to find E[Y].\n",
    "\n",
    "(c) Check your answer in (b) by computing E[Y] directly without using the marginal\n",
    "PDF of Y."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SElwmPGfOh5r"
   },
   "source": [
    "Ans:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RnxklZ43q8Xp"
   },
   "source": [
    "---\n",
    "**Q9**\n",
    "\n",
    "Problem 18. Let X be a random variable with PDF\n",
    "\n",
    "$f_X(x)= x/4$ if $1 < x \\leq 3$\n",
    "\n",
    "$\\ \\ \\ = 0$, otherwise,\n",
    "\n",
    "and let A be the event ${X \\geq 2}$.\n",
    "\n",
    "\n",
    "(a) Find E[X],P(A),$f_{X|A}(x)$,and $E[X|A]$.\n",
    "\n",
    "(b) Let $Y=X^2$. Find $EY$ and $Var(Y)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tX9qAugysKP0"
   },
   "source": [
    "Ans:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gv4Tn89U5HZ_"
   },
   "source": [
    "---\n",
    "**Q10**\n",
    "\n",
    "Problem21. We start with a stick of length $l$. We break it at a point which is chosen according to a uniform distribution and keep the piece, of length Y, that contains the left end of the stick. We then repeat the same process on the piece that we were left with, and let X be the length of the remaining piece after breaking for the second time.\n",
    "\n",
    "(a) Find the joint PDF of Y and X.\n",
    "\n",
    "(b) Find the marginal PDF of X.\n",
    "\n",
    "(c) Use the PDF of X to evaluate E[X].\n",
    "\n",
    "(d) Evaluate E[X], by exploiting the relation $X =Y\\cdot(X/Y)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_AkWk0tn5hxv"
   },
   "source": [
    "Ans:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R-f6Y0mv_Rk-"
   },
   "source": [
    "---\n",
    "**Q11**\n",
    "\n",
    "Problem 25. The coordinates X and Y of a point are independent zero mean normal random variables with common variance $\\sigma^2$. Given that the point is at a distance of at least c from the origin. find the conditional joint PDF of X and Y."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QdUbtZHu_kUB"
   },
   "source": [
    "Ans:\n",
    "\n",
    "$X \\sim \\mathcal{N}(0, \\sigma^2)$ and $Y \\sim \\mathcal{N}(0, \\sigma^2)$\n",
    "$\n",
    "\\begin{aligned}\n",
    "\\implies\n",
    "f_X (x)\n",
    "& = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-\\frac{x^2}{2\\sigma^2}}\n",
    ", \\quad\n",
    "f_Y (y)\n",
    "& = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-\\frac{y^2}{2\\sigma^2}}\n",
    "\\end{aligned}\n",
    "$\n",
    "\n",
    "Given that the point is at a distance of at least c from the origin, we have $X^2 + Y^2 \\geq c^2$.\n",
    "\n",
    "Let $C = \\{(x, y) | x^2 + y^2 \\geq c^2\\}$.\n",
    "\n",
    "$\n",
    "\\begin{aligned}\n",
    "\\implies\n",
    "f_{X, Y | C} (x, y)\n",
    "&\n",
    "= \\frac{f_{X, Y} (x, y)}{P[C]}\n",
    "= \\frac{f_X (x) f_Y (y)}{P[C]},\n",
    "\\quad\n",
    "\\because X, Y \\text{ are independent}\n",
    "\\\\\n",
    "\\implies\n",
    "P[C]\n",
    "&\n",
    "= \\int_{-\\infty}^{\\infty} \\int_{-\\infty}^{\\infty} f_X (x) f_Y (y) \\mathbb{I}_{\\{x^2 + y^2 \\geq c^2\\}} dx dy\n",
    "= \\int_{-\\infty}^{\\infty} \\int_{-\\infty}^{\\infty} \\frac{1}{\\sigma^2 2\\pi} e^{-\\frac{x^2 + y^2}{2\\sigma^2}} \\mathbb{I}_{\\{x^2 + y^2 \\geq c^2\\}} dx dy\n",
    "\\\\ &\n",
    "= \\frac{1}{\\sigma^2 2\\pi} \\int_{0}^{2\\pi} \\int_{0}^{\\infty} e^{-\\frac{r^2}{2\\sigma^2}} \\mathbb{I}_{\\{r \\geq c\\}} r dr d\\theta\n",
    "= \\frac{1}{\\sigma^2 2\\pi} \\int_{0}^{2\\pi} \\int_{c}^{\\infty} e^{-\\frac{r^2}{2\\sigma^2}} r dr d\\theta\n",
    "\\\\\n",
    "\\implies\n",
    "\\int_{c}^{\\infty} e^{-\\frac{r^2}{2\\sigma^2}} r dr\n",
    "& = \\frac{1}{2} \\int_{c^2}^{\\infty} e^{-\\frac{t}{2\\sigma^2}} dt\n",
    "= \\frac{1}{2} \\left[ -2\\sigma^2 e^{-\\frac{t}{2\\sigma^2}} \\right]_{c^2}^{\\infty}\n",
    "= \\sigma^2 e^{-\\frac{c^2}{2\\sigma^2}}\n",
    "\\\\\n",
    "\\implies\n",
    "P[C]\n",
    "&\n",
    "= \\frac{1}{\\sigma^2 2\\pi} \\int_{0}^{2\\pi} \\sigma^2 e^{-\\frac{c^2}{2\\sigma^2}} d\\theta\n",
    "= \\frac{1}{2\\pi} e^{-\\frac{c^2}{2\\sigma^2}} \\int_{0}^{2\\pi} d\\theta\n",
    "= e^{-\\frac{c^2}{2\\sigma^2}}\n",
    "\\\\\n",
    "\\implies\n",
    "f_X (x) f_Y (y)\n",
    "&\n",
    "= \\frac{1}{\\sigma^2 2\\pi} e^{-\\frac{x^2 + y^2}{2\\sigma^2}}\n",
    "\\\\\n",
    "\\implies\n",
    "f_{X, Y | C} (x, y)\n",
    "&\n",
    "= \\frac{f_X (x) f_Y (y)}{P[C]}\n",
    "= \\frac{1}{\\sigma^2 2\\pi} e^{-\\frac{x^2 + y^2}{2\\sigma^2} + \\frac{c^2}{2\\sigma^2}}\n",
    "= \\frac{1}{\\sigma^2 2\\pi} e^{-\\frac{(x^2 + y^2 - c^2)}{2\\sigma^2}}\n",
    "\\end{aligned}\n",
    "$\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "\\implies\n",
    "\\boxed{\n",
    "f_{X, Y | C} (x, y)\n",
    "= \\frac{1}{\\sigma^2 2\\pi} e^{-\\frac{(x^2 + y^2 - c^2)}{2\\sigma^2}}\n",
    "}\n",
    "$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NN9ilzMnv4tw"
   },
   "source": [
    "---\n",
    "**Q12**\n",
    "\n",
    "Problem 34. A defective coin minting machine produces coins whose probability of heads is a random variable P with PDF\n",
    "\n",
    "$f_P(p) = pe^p$, if $p \\in [0,1]$,\n",
    "\n",
    "$\\ \\ \\ =0$, otherwise.\n",
    "\n",
    "A coin produced by this machine is selected and tossed repeatedly, with successive tosses assumed independent.\n",
    "\n",
    "(a) Find the probability that a coin toss results in heads.\n",
    "\n",
    "(b) Given that a coin toss resulted in heads, find the conditional PDF of P.\n",
    "\n",
    "(c) Given that the first coin toss resulted in heads, find the conditional probability of heads on the next toss."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5S7ZnKDiwOUx"
   },
   "source": [
    "Ans:\n",
    "\n",
    "We're given a random variable $P$ denoting the probability of heads on a coin toss, with PDF\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "f_P (p)\n",
    "= \\begin{cases}\n",
    "pe^p, & \\text{if } p \\in [0, 1]\\\\\n",
    "0, & \\text{otherwise}\n",
    "\\end{cases}\n",
    "$\n",
    "\n",
    "We can see that this a valid PDF by\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "\\int_{-\\infty}^{\\infty} f_P (p) dp\n",
    "= \\int_{0}^{1} f_P (p) dp\n",
    "= \\int_{0}^{1} pe^p dp\n",
    "= \\left[ p e^p - \\int e^p dp \\right]_{0}^{1}\n",
    "= \\left[ p e^p - e^p \\right]_{0}^{1}\n",
    "= (e - e) - (0 - 1)\n",
    "= 1\n",
    "$\n",
    "\n",
    "Now, we're also given that successive coin tosses are independent.\n",
    "\n",
    "(a)\n",
    "Let $H$ denote the event that a coin toss results in heads.\n",
    "Then, the probability that a coin toss results in heads is given by\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "\\Pr(H)\n",
    "= E[P]\n",
    "= \\int_{-\\infty}^{\\infty} p f_P (p) dp\n",
    "= \\int_{0}^{1} p^2 e^p dp\n",
    "= \\left[ p^2 e^p - \\int 2p e^p dp \\right]_{0}^{1}\n",
    "= \\left[ p^2 e^p \\right]_{0}^{1} - 2 \\int_{0}^{1} p e^p dp\n",
    "= e - 2\n",
    "$\n",
    "\n",
    "Hence, the probability that a coin toss results in heads is $\\boxed{\\Pr(H) = e - 2 \\approx 0.72}$.\n",
    "\n",
    "(b)\n",
    "Given that a coin toss resulted in heads, the conditional PDF of $P$ is given by\n",
    "\n",
    "$\n",
    "\\displaystyle\n",
    "f_{P|H} (p)\n",
    "= \\frac{f_P (p)}{f_H (H)}\n",
    "= \\frac{pe^p}{e}\n",
    "= pe^{p-1}\n",
    "$\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}