analysis-I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysis Part I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from cibin import tau_twosided_ci\n",
    "from cibin import hypergeom_conf_interval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One-sided confidence intervals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Upper one-sided bounds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regeneron data\n",
    "n=753\n",
    "m=752\n",
    "N=n+m\n",
    "n01 = 59\n",
    "n11 = 11\n",
    "n00 = m-n01\n",
    "n10 = n-n11\n",
    "alpha=0.05"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-1.0, 0.6737541528239203]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve upper one-sided bounds via simultaneous Bonferroni confidence bounds for N_+1 and N_1+ \n",
    "Ndot_1 = hypergeom_conf_interval(n11*N/n, n11, N, 1-alpha/2, alternative=\"upper\")\n",
    "N1_dot = hypergeom_conf_interval(n01*N/70, n01, N, 1-alpha/2, alternative=\"lower\")\n",
    "upp = (Ndot_1[1] - N1_dot[0])/N\n",
    "lower = (Ndot_1[0] - N1_dot[1])/N\n",
    "ci = [lower, upp]\n",
    "ci"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Lower one-sided bounds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.23255813953488372, 1.0]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve lower one-sided bounds via simultaneous Bonferroni confidence bounds for N_+1 and N_1+\n",
    "Ndot_1 = hypergeom_conf_interval(n11*N/n, n11, N, 1-alpha/2, alternative=\"lower\")\n",
    "N1_dot = hypergeom_conf_interval(n01*N/70, n01, N, 1-alpha/2, alternative=\"upper\")\n",
    "upp = (Ndot_1[1] - N1_dot[0])/N\n",
    "lower = (Ndot_1[0] - N1_dot[1])/N\n",
    "ci = [lower, upp]\n",
    "ci"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two-sided confidence intervals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Two-sided bounds with Sterne method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.69375415, 0.2123581]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve two-sided bounds via Sterne's method\n",
    "Ndot_1 = hypergeom_conf_interval(round(n11*N/n), n11, N, 1-alpha, alternative=\"two-sided\")\n",
    "N1_dot = hypergeom_conf_interval(round(n01*N/70), n01, N, 1-alpha, alternative=\"two-sided\")\n",
    "lower = (Ndot_1[0] - N1_dot[1])/N\n",
    "upp = (Ndot_1[1] - N1_dot[0])/N\n",
    "ci=[lower,upp]\n",
    "ci"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Two-sided bounds with Li and Ding's method 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-0.15813, 0.4279]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve two-sided bounds via method 3 in Li and Ding\n",
    "ci = tau_twosided_ci(n11, n10, n01, n00, 0.05,exact=False, reps=1)[0]\n",
    "ci"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference between the two-sided bounds are the coverage region of the result. From hypergeometric distribution with sterne method. The lower bound is postive, and the lower bound for method 3 is negative and the upper bound is larger for sterne is larger than the upper bound of method 3. The conclusion might be that the treatment effect might not have such good performance if we use method 3 to retrive to result, which means the press might assume the result to be overestimate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Legitimate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is legitimate to use one-sided confidence  intervals if the outcome only leaves out one side of the result, depending on the type of data we are looking at. For treatment effect, it is not reasonable to have only one side of cumulative distribution, since the one-sided confience interval might not contain the outcome whether it is upper or lower one-sided. And it is not reasonable to have extreme value such as 1 and -1 in 95% confidence intervals. The average effect should be zero to fulfill the null hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preferable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is preferable to use two-sided confidence interval since one-sided intervals might accept extreme outcome or reject outcome that are acceptable if it is two-sided confidence interval. And also for treatment effect, it is reasonable that average confidence intervals contain 1-alpha probabilities by leaving out two sides of result since we can not guarantee that the result of the treatment will always have the worst performance or the best performance from the confidence intervals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}