01_jupyter_howto.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What is Jupyter and how do you use it?\n",
    "\n",
    "[Project Jupyter](http://jupyter.org) exists to develop software and standards for interactive computing. Jupyter, which you are using now, provides an interactive notebook which runs on a cloud-server, and with which you interact using a web browser.\n",
    "\n",
    "As installed, Jupyter provides two things:\n",
    "\n",
    "* An interactive Python notebook, which you are using now (this is an interactive notebook!)\n",
    "* An interactive bash terminal (we will use this later)\n",
    "\n",
    "What is an interactive Python notebook? It is a notebook that mixes documentation (like this!) with Python code (like below), with the output of that code.\n",
    "\n",
    "For example, below is the Python code to print \"Hello World\". Click on the below code and then press \"SHIFT+Return\". You should see that the code is run, and below \"Hello World\" will be printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Hello World\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above code is interactive because it runs when you pressed \"SHIFT+Return\". It is also interactive because you can change it. For example, click again on the code and change \"Hello World\" to \"Hello Jupyter\". Press \"SHIFT+Return\" again and you should see \"Hello Jupyter\" printed.\n",
    "\n",
    "A code cell can have as much or little Python in it as you want. The below cell defines a function and then runs it in a loop. What do you think will be printed out when you select it and press \"SHIFT+Return\"? Have a go. Are you right?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def countDown(start):\n",
    "    for i in range(start, 0, -1):\n",
    "        print(\"%d...\" % i)\n",
    "    print(\"Lift off!\")\n",
    "    \n",
    "countDown(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each cell can use the variables and functions defined in previous cells. For example, the next cell sets the value of the variable `x` to 5, while the following cell then uses that to call the `countDown` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countDown(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can go back up and edit the value of `x`. For example, set `x` to 7 and press SHIFT+Return. This has changed `x`. However, it doesn't re-run any code that depends on `x`. That is for you to do. You need to click on the next cell `countDown(x)` and press SHIFT+Return again. You should then see that the code is run and updated.\n",
    "\n",
    "Because you can change any cell at any time, a notebook can get into a confused state. If we change `x` again to `21`, then the cell `countDown(x)`, is now out of date, as it depends on the old value of `x`. So, how can we know what state each cell is in? The state is based on the order in which cells are executed. This is shown by the little number next to each cell, e.g. see below.\n",
    "\n",
    "![Image showing Jupyter state](images/jupyter_state.jpeg)\n",
    "\n",
    "The number \"10\" shows that the line `x = 21` was the 10th cell executed by Python, while the line `countDown(x)` was the 9th cell executed. As `countDown(x)` was executed before `x = 21`, it was not affected by `x = 21`. If you want it to be affected, you need to execute that cell again (so it is now the 11th cell executed), e.g.\n",
    "\n",
    "![Image showing updated Jupyer state](images/jupyter_state2.jpeg)\n",
    "\n",
    "If you find yourself getting confused, then click \"Kernel | Restart & Clear Output\" from the menu above to clear all outputs, clear the state, and reset back to a clean notebook.\n",
    "\n",
    "![Image showing clear and restart](images/jupyter_restart.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interactive Graphics\n",
    "\n",
    "In addition to mixing documentation with interactive Python, the notebook also allows Python modules to embed interactive graphics. For example, the Python molecular viewer [nglview](https://github.com/arose/nglview) can be used to view molecules from within a notebook.\n",
    "\n",
    "Execute the cell below by selecting it and pressing SHIFT+Return. This will import NGLView, load the molecules contained in the file `data/complex.pdb` and display it below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "542c63f3308f423ea336bc210f8a07e2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>NGLWidget</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "NGLWidget()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import nglview as nv\n",
    "view = nv.show_file(\"data/complex.pdb\")\n",
    "view"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can rotate and interact with molecules once they have loaded (the \"[*]\" next to a cell indicates that it is processing. It will turn into a number once processing is complete)\n",
    "\n",
    "You can change molecular representations and interact with the 3D molecule view. [Later on](16_viewing_molecules.ipynb) we will go through some examples showing what you can do and how you can select parts of the molecule (or [click here](16_viewing_molecules.ipynb) if you want to jump ahead and learn now).\n",
    "\n",
    "In addition to viewing molecules, you can also draw graphs. This is achieved using a combination of the [matplotlib](https://matplotlib.org), [numpy](http://www.numpy.org) and [pandas](https://pandas.pydata.org) modules. For example, the below code draws a sine and cosine curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, import pandas, numpy and matplotlib\n",
    "import pandas as pd\n",
    "from pandas import Series, DataFrame\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%config InlineBackend.figure_format = 'svg'   # helps make things look better in Jupyter :-)\n",
    "\n",
    "# now define a sine and cosine curve\n",
    "X = np.linspace(-np.pi, np.pi, 256, endpoint=True)\n",
    "data = {'cos': np.cos(X), 'sin': np.sin(X)}\n",
    "trig = DataFrame(index=X, data=data)\n",
    "\n",
    "trig.plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Later on we will go through [pandas](12_pandas.ipynb), [matplotlib](15_matplotlib.ipynb) and [numpy](13_basic_numpy.ipynb) and you will learn how to draw lots of different and interesting graphs (or click the links to jump ahead and learn now)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mixing documentation and code\n",
    "\n",
    "In addition to interactivity, the real benefit of a Jupyter notebook is that it encourages you to include lots of documentation with your code. In effect, a notebook is a combination of code, documentation, and (once it has been run) analysis and data.\n",
    "\n",
    "You can create documentation by changing the type of a cell. The type of a cell is indicated in the drop-down at the top, under the menu. If you click on the graph above, you will see that the type of a code cell is \"Code\".\n",
    "\n",
    "![Image showing code type](images/jupyter_type_code.jpeg)\n",
    "\n",
    "If you now click on this documentation, you should see the cell type change to \"Markdown\"\n",
    "\n",
    "![Image showing markdown type](images/jupyter_type_markdown.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating your own cells\n",
    "\n",
    "You can create your own cells in two different ways;\n",
    "\n",
    "* either by pressing SHIFT+Return on the last cell (which creates a new cell at the bottom of the notebook),\n",
    "* or by clicking \"Insert | Cell Above\" or \"Insert | Cell Below\" from the menu to insert a new cell above or below the currently selected cell.\n",
    "\n",
    "You can choose whether this should be a code or documentation (markdown) cell by selecting the appropriate value from the drop-down at the top.\n",
    "\n",
    "Code cells can contain any valid Python. Documentation (markdown) cells can contain any text. The text can be formatted, using the [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) standard.\n",
    "\n",
    "Have a go by creating some new documentation and code cells below. If you want some inspiration for documentation, take a look at the documentation cells above by double-clicking on them. You should see how I have inserted images, lists, headings etc.\n",
    "\n",
    "You can then render the documentation using SHIFT+Return (just like running a code cell)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Controlling the Python kernel\n",
    "\n",
    "The Python process that is interpreting and executing your Python code is referred to as the \"kernel\" of the notebook (hence the \"Kernel\" menu above). When you execute a code cell it sends the code to the Python kernel, and then waits for the kernel to send a result back. The Python kernel can only execute one thing at a time. This means that, for slow functions, you could be executing cells too quickly and leave the Python kernel behind.\n",
    "\n",
    "For example, here is a function that fakes a slow function as it goes to sleep for the specified number of seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sleep(n):\n",
    "    import os\n",
    "    os.system(\"sleep %d\" % n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we will have three code cells that each sleep for a long time. Execute all three quickly one after another..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sleep(30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sleep(30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sleep(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see that there is a [*] next to each code cell. This is because they are all waiting for the Python kernel to finish executing and return.\n",
    "\n",
    "![Jupyter is sleeping](images/jupyter_sleep.jpeg)\n",
    "\n",
    "If you wait you will see each sleep finishing and control then passing to the next.\n",
    "\n",
    "Sometimes you don't want to be patient, and you want to stop the Python kernel from running a function. To stop (interupt) the kernel, either click the \"Stop\" icon at the top of the screen, or select \"Kernel | Interupt\" from the menu.\n",
    "\n",
    "![Jupyter stop button](images/jupyter_stop.jpeg)\n",
    "\n",
    "Other useful buttons up there are \"restart the kernel\" (same as the menu Kernel | Restart & Clear Output) and the arrows that can move the selected cell up and down the notebook.\n",
    "\n",
    "Have a play. Explore the buttons and menu items. Take a look at the \"Help\" in the menu. Remember that, if everything goes wrong, you can always click \"Stop\" or select \"Kernel | Restart & Clear Output\" from the menu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running external programs in a code cell\n",
    "\n",
    "You can do more than just execute Python in the code cells. You can run *any command* you want! Just put an exclamation mark first, and then type the command normally as you would in a terminal. For example, the below cell is running `ls`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can run any command that you want, exactly as you would typing in a terminal. For example, lets use `cat` to read the LICENSE file..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cat LICENSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can work with directories as well. For example, here we will copy the `dioxin.pdb` file from `data` into the current directory and will then view it using viewMolecules..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cp data/dioxin.pdb ./"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nglview as nv  # importing again in case we restarted the kernel\n",
    "view =nv.show_file(\"dioxin.pdb\")\n",
    "view"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Saving the notebook\n",
    "\n",
    "At the top of the screen is the name of the notebook, and its current save state.\n",
    "\n",
    "![Jupyter file info](images/jupyter_files.jpeg)\n",
    "\n",
    "The current save state tells you when the last checkpoint was saved, and whether or not the notebook contains any unsaved changes. Checkpoints provide points in the notebook that you can refer back to, using the menu item \"File | Revert to Checkpoint\". You are always able to revert to the last checkpoint, meaning that if you make a mistake, you can go back to a previous safe version.\n",
    "\n",
    "If you click \"File | Save and Checkpoint\", or click the \"Save\" icon (floppy disk icon on the far left), then the current state of the notebook will be saved. This includes not just the code and documentation you have written, but also all of the outputs and graphs that you have created. This is really useful, as it lets you save your work, and then come back later.\n",
    "\n",
    "For example, why don't you save now, and then close this window and then open the notebook again? You should see that everything you have done, and everything that was output is restored. Did it work?\n",
    "\n",
    "The notebook was saved to a file called `01_jupyter_howto.ipynb`. The `.ipynb` signifies that the file is a Jupyter notebook (formerly called an interactive python notebook). You can change the name by clicking on the name of the notebook above, and then choosing a different name in the window that opens.\n",
    "\n",
    "Alternatively, you can save a copy of this notebook under a different name by clicking \"File | Make a Copy...\" in the menu. You can then copy this file or send it to someone else. As a notebook contains the code, data, outputs and analysis, sharing notebooks is the closest thing we have to interactive papers. This is a really powerful concept and, I believe, will fundementally change how we report and share the results of computational and data analysis in science."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Jupyter Hub\n",
    "\n",
    "When you first logged into this website you launched this notebook by clicking on its name in the Jupyter Hub. This was the page that looked like a file browser and that (hopefully) is still open in one of the tabs in your browser. The Jupyter Hub is not just a file brower. It is the gateway to do a lot more than just interact with a single notebook.\n",
    "\n",
    "The first thing to note is that this Jupyter Hub is running on the cloud and consumes resources. It is important that you click \"Logout\" (top right) when you have finished, as this will free up the hub to be used by someone else. There is a timer that will automatically log you out 40 minutes after your web browser is closed. However, please remember to log out if you can.\n",
    "\n",
    "![Jupyter hub](images/jupyter_hub.jpeg)\n",
    "\n",
    "The main view of the hub is the file browser that shows all of your files. You can navigate through the files and directories like any normal file browser. For example, you can see above the `01_jupyter_howto.ipynb` file that contains this notebook. You should see that the file has a green icon next to it, as well as a `Running` indicator on the right. This indicates that the notebook is currently running.\n",
    "\n",
    "Running notebooks consume cloud resources, which are limited. You can only have 1.8 GB of data in memory and have limited CPU. You should thus shutdown any running notebooks that you aren't using any more. To do this, click the checkbox to the left of the notebook filename. This will bring up some buttons at the top of the screen. Click the \"Shutdown\" button to shut down the notebook.\n",
    "\n",
    "![Shutdown running kernels](images/jupyter_shutdown.jpeg)\n",
    "\n",
    "Alternatively, you can go to the \"Running\" tab to see everything of your's that is currently running in the hub. Click the \"Shutdown\" button on the right for anything that you aren't using and want to shut down.\n",
    "\n",
    "![Running tab](images/jupyter_running.jpeg)\n",
    "\n",
    "Back in the \"Files\" tab, clicking the checkbox next to a file brings up buttons that let you download files. You can use the hub to download files by selecting them and clicking the \"Download\" button that will appear. Other buttons will appear depending on the type of file, e.g. \"Duplicate\", \"View\", \"Edit\" etc. The bin icon will delete the file.\n",
    "\n",
    "![Download a file](images/jupyter_download.jpeg)\n",
    "\n",
    "You can create new notebooks or files by clicking on the \"New\" button on the right. This will open a drop-down menu for you to select the type of thing to create.\n",
    "\n",
    "![New dropdown](images/jupyter_new.jpeg)\n",
    "\n",
    "As well as letting you create a new jupyter notebook (by clicking \"New | Python3\"), you can also create new text files (\"New | Text File\") or new folders (\"New | Folder\"). Most interestingly, is that you can also create a new bash terminal. If you click \"New | Terminal\" it will open a new tab in which you will have a fully functional bash terminal shell. This is running on the same cloud server as your notebook. It is 100% completely a fully functional terminal within your web browser. You could even use it to run the normal command-line python ;-)\n",
    "\n",
    "![Bash in the browser](images/jupyter_shell.jpeg)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}