TB_or_not_TB.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4a240127-3977-4094-b983-e867907d7832",
   "metadata": {},
   "source": [
    "# TB or not TB\n",
    "## That is the question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3be6b3bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip freeze > requirements_1.txt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b5c6dc5b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python 3.7.10\r\n"
     ]
    }
   ],
   "source": [
    "! python --version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "39756d97-0268-4e9c-8fec-7c69a8fb7c63",
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.vision.all import *\n",
    "from fastai.vision.widgets import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "be09ecc4-075b-4750-800b-bcd3de097054",
   "metadata": {},
   "outputs": [],
   "source": [
    "defaults.device = torch.device('cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c1dcfb6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "def label_func(fname):\n",
    "    return \"Normal\" if bool(re.search(r'\\w{3}CXR_\\d+_0',fname.name)) == True else \"TB\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4c761723-8ab3-4aed-bf3b-20fbd4af236e",
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Path()\n",
    "learn_inf = load_learner(path/'resent50_cutmix_6.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e0881b22-c4f8-4192-bd5c-dd97698ea673",
   "metadata": {},
   "outputs": [],
   "source": [
    "btn_upload = widgets.FileUpload()\n",
    "out_pl = widgets.Output()\n",
    "lbl_pred = widgets.Label()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f3d71f7c-b735-4df9-a7c5-2bb6f1348d18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create interface\n",
    "\n",
    "def on_data_change(change):\n",
    "    lbl_pred.value = ''\n",
    "    \n",
    "    # Load Image\n",
    "    img = PILImage.create(btn_upload.data[-1])\n",
    "    out_pl.clear_output()\n",
    "    with out_pl:\n",
    "        display(img.to_thumb(224,244))\n",
    "    \n",
    "    # Predict image\n",
    "    pred, pred_idx, probs = learn_inf.predict(img)\n",
    "    \n",
    "    lbl_pred.value = f'Prediction: {pred}; Probability: {probs[pred_idx]:.04f}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "14419cde",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0874da02fbc544d69a3ff4b8a8f9a32c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Label(value='Select your xray!'), FileUpload(value={'TRAIN_px47.jpg': {'metadata': {'name': 'TR…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "btn_upload.observe(on_data_change, names=['data'])\n",
    "\n",
    "# Display Predictions\n",
    "display(VBox([widgets.Label('Select your xray!'), btn_upload, out_pl, lbl_pred]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc6de4ad",
   "metadata": {},
   "source": [
    "This is a Deep learning powered app that detects TB from chest X rays. The code and data reside at: [Tb Or Not TB repository](https://github.com/aflip/tb-or-not-tb/)\n",
    "\n",
    "## Specs\n",
    "\n",
    "This uses a naive Resnet50 RNN [8] and achieves the following metrics on the validation data. This is a work in progress and I will be testing performance on various datasets.\n",
    "\n",
    "Sensitivity: 88.506,\n",
    "Specificity: 82.192, \n",
    "Precision: 85.555,\n",
    "Accuracy: 85.625,\n",
    "\n",
    "Which seems to be slightly better than human performance. \n",
    "\n",
    "\n",
    "Made with Fast.ai and trained on the the Montgomery and Shenzhen TB datasets and tested on the Jaypee institute TB Dataset [5]\n",
    "\n",
    "The app is strictly not for diagnosis or treatment or making medical decisions as it has not undergone any clinical testing yet.\n",
    "\n",
    "## How reliable is the Chest X-ray (CXR)\n",
    "\n",
    "In practice, the CXR is used widely to make a confirmation of TB. It is also considered to be very sensitive for TB. Probably more sensitive than just a sputum test. However, the inter-operator variability in CXR reading is a big problem in TB. Studies indicate human sensitivity and specificity ranges from 72-85% [3] And also that inter operator variability might be as high as 29% [4] \n",
    "\n",
    "\n",
    "Multiple studies have shown that DL models provide consistent and reliable TB diagnosis from CXRs [6][7]\n",
    "\n",
    "I think removing the inter-operator bias, and Dl models that provide more than just labels with confidence, but also include localization, heat-maps or boundary boxes will go a long way in improving the diagnosis making process of TB.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## References:\n",
    "\n",
    "[1] @misc{howard2018fastai, title={fastai}, author={Howard, Jeremy and others}, year={2018}, publisher={GitHub}, howpublished={\\url{https://github.com/fastai/fastai}}, }\n",
    "\n",
    "[2] Jaeger, S., Candemir, S., Antani, S., Wáng, Y. X., Lu, P. X., & Thoma, G. (2014). Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quantitative imaging in medicine and surgery, 4(6), 475–477. https://doi.org/10.3978/j.issn.2223-4292.2014.11.20\n",
    "\n",
    "[3] World Health Organization. (‎2016)‎. Chest radiography in tuberculosis detection: summary of current WHO recommendations and guidance on programmatic approaches. World Health Organization. https://apps.who.int/iris/handle/10665/252424\n",
    "\n",
    "[4] Toman, K. (2004). Toman's Tuberculosis: case detection, treatment, and monitoring: questions and answers. World Health Organization.\n",
    "\n",
    "[5] Chauhan A, Chauhan D, Rout C (2014) Role of Gist and PHOG Features in Computer-Aided Diagnosis of Tuberculosis without Segmentation. PLoS ONE 9(11): e112980. https://doi.org/10.1371/journal.pone.0112980\n",
    "\n",
    "[6] Harris, M., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Ahmad Khan, F. (2019). A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. PloS one, 14(9), e0221339. https://doi.org/10.1371/journal.pone.0221339\n",
    "\n",
    "[7] Qin, Z. Z., Sander, M. S., Rai, B., Titahong, C. N., Sudrungrot, S., Laah, S. N., ... & Creswell, J. (2019). Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems. Scientific reports, 9(1), 1-10.\n",
    "\n",
    "[8] @article{He2015,\n",
    "\tauthor = {Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun},\n",
    "\ttitle = {Deep Residual Learning for Image Recognition},\n",
    "\tjournal = {arXiv preprint arXiv:1512.03385},\n",
    "\tyear = {2015}\n",
    "}\n",
    "\n",
    "\n",
    "## PS:\n",
    "\n",
    "Whether 'tis nobler in the mind to suffer   \n",
    "The slings and arrows of outrageous fortune,   \n",
    "Or to take arms against a sea of troubles   \n",
    "And by opposing end them. To die—to sleep,   \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}