Skip to content

Commit

Permalink
Merge pull request #187 from akabe/feature/magic
Browse files Browse the repository at this point in the history 
Utility functions inspired by IPython magics
  • Loading branch information
@akabe
akabe authored Apr 10, 2022
2 parents 7ea00fd + b28da24 commit 26d50e6
Show file tree
Hide file tree
Showing 19 changed files with 1,617 additions and 23 deletions.
1 change: 1 addition & 0 deletions .github/workflows/ci.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ jobs:
- name: Install dependencies
run: |
sudo apt-get install -y bash zsh python2 python3 ruby
opam install . -y --deps-only --with-test
opam install 'merlin>3.0.0' -y
Expand Down
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ OCaml Jupyter can be installed by [OPAM][opam] as follows:
pip install jupyter
opam install jupyter
grep topfind ~/.ocamlinit || echo '#use "topfind";;' >> ~/.ocamlinit # For using '#require' directive
grep Topfind.log ~/.ocamlinit || echo 'Topfind.log:=ignore;;' >> ~/.ocamlinit # Suppress logging of topfind (recommended but not necessary)
ocaml-jupyter-opam-genspec
jupyter kernelspec install [ --user ] --name "ocaml-jupyter-$(opam var switch)" "$(opam var share)/jupyter"
```
Expand Down
1 change: 1 addition & 0 deletions jupyter.opam
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ depends: [
"zmq-lwt" {>= "5.0.0"}
"yojson" {>="1.6.0"}
"ppx_yojson_conv" {>= "0.14.0"}
"ppx_deriving" {>= "5.2.1"}
"cryptokit" {>= "1.12"}
"dune" {>= "1.0.0"}
"ounit2" {with-test & >= "2.0.0"}
Expand Down
374 changes: 374 additions & 0 deletions notebooks/magics.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,374 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "hundred-appearance",
"metadata": {},
"source": [
"# Utility functions inspired by IPython magics\n",
"\n",
"[IPython magic](https://ipython.readthedocs.io/en/stable/interactive/magics.html) is special command available in IPython kernel.\n",
"The magic provides helpful functionality such as calling external commands (bash, ruby, etc.), benchmarking, capturing of stdout/stderr, and so on.\n",
"The syntax of the magics is different from the Python core language: they starts with `%` or `%%`, and arguments are written like `--param`.\n",
"\n",
"OCaml Jupyter 2.8.0+ supports several utility functions inspired by IPython magics.\n",
"Unlike IPython magics, the functions are implemented by the OCaml core language, not syntax extension.\n",
"You don't need to study new syntax, and distinguish cell magics and line magics.\n",
"The utility functions inherits convenience and flexibility of the OCaml language.\n",
"They are included in `jupyter.notebook` package. You can see [API documentation of jupyter.notebook](https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/) for details.\n",
"\n",
"For example, you can measure execution time of a snippet by the cell magic `%%timeit` in IPython as follows.\n",
"\n",
"```python\n",
"%%timeit\n",
"2 ** 1000\n",
"# > 884 ns ± 15.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n",
"```\n",
"\n",
"In OCaml Jupyter, `Jupyter_notebook.Bench.timeit` has the same functionality as `%%timeit`:\n",
"\n",
"```ocaml\n",
"open Jupyter_notebook\n",
"Bench.timeit (fun () -> 2. ** 1000.)\n",
"(* > - : Jupyter_notebook.Bench.stat Jupyter_notebook.Bench.t =\n",
" * > Jupyter_notebook.Bench.({\n",
" * > b_rtime = 25.639 ns ± 3.523 ns; b_utime = 25.518 ns ± 3.486 ns;\n",
" * > b_stime = 0.053 ns ± 0.037 ns; }) *)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "curious-reality",
"metadata": {
"scrolled": true
},
"outputs": [
],
"source": [
"#require \"jupyter.notebook\" ;;\n",
"open Jupyter_notebook ;;"
]
},
{
"cell_type": "markdown",
"id": "enclosed-resort",
"metadata": {},
"source": [
"## 1. Benchmark\n",
"\n",
"Benchmark functions are in `Jupyter_notebook.Bench`.\n",
"We provide `time` and `timeit` corresponding to `%%time` and `%%timeit`, respectively.\n",
"\n",
"[Bench.time](https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/Bench/index.html#val-time) evaluates a function once, and reports time of the execution.\n",
"The first component of a return value is `f ()`, and the second is benchmark result:\n",
"\n",
"- `b_rtime` is real time\n",
"- `b_utime` is user time\n",
"- `b_stime` is sys time"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "intended-neighborhood",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"- : float * float Jupyter_notebook.Bench.t =\n",
"(1.07150860718626732e+301, Jupyter_notebook.Bench.({\n",
" b_rtime = 4.053 us; b_utime = 2.000 us; b_stime = 3.000 us; }))\n"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Bench.time (fun () -> 2. ** 1000.)"
]
},
{
"cell_type": "markdown",
"id": "unauthorized-pasta",
"metadata": {},
"source": [
"[Bench.timeit](https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/Bench/index.html#val-timeit) is very similar to `Bench.time`, but the former calls a function repeatedly, and calculates mean of execution time of them.\n",
"`timeit` is a little reliable then `time`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "descending-ancient",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Bench.stat Jupyter_notebook.Bench.t =\n",
"Jupyter_notebook.Bench.({\n",
" b_rtime = 26.035 ns ± 3.516 ns; b_utime = 25.773 ns ± 3.352 ns;\n",
" b_stime = 0.132 ns ± 0.081 ns; })\n"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Bench.timeit (fun () -> 2. ** 1000.)"
]
},
{
"cell_type": "markdown",
"id": "hearing-communist",
"metadata": {},
"source": [
"## 2. Subprocess\n",
"\n",
"### 2.1. Command bindings\n",
"\n",
"You can call commands and executable programs from OCaml Jupyter.\n",
"[Jupyter_notebook.Process](https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/Process/index.html) contains easy-to-use bindings of some popular commands.\n",
"\n",
"For example, `ls` command corresponds to `Process.ls`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "monthly-eight",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Untitled.ipynb\n",
"datasets\n",
"install_ocaml_colab.ipynb\n",
"introduction.ipynb\n",
"utility_like_ipython_magic.ipynb\n",
"word_description_from_DuckDuckGoAPI.ipynb\n"
]
},
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Process.t =\n",
"{Jupyter_notebook.Process.exit_status = Unix.WEXITED 0; stdout = None;\n",
" stderr = None}\n"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Process.ls [\".\"]"
]
},
{
"cell_type": "markdown",
"id": "demonstrated-pathology",
"metadata": {},
"source": [
"`sh` is also available."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "collective-alberta",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[NOTE] It seems you have not updated your repositories for a while. Consider updating them with:\n",
" opam update\n",
"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"The following actions will be performed:\n",
" ∗ install conf-lapack 1 [required by lacaml]\n",
" ∗ install conf-blas 1 [required by lacaml]\n",
" ∗ install lacaml 11.0.8\n",
"===== ∗ 3 =====\n",
"\n",
"<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><> 🐫 \n",
"[lacaml.11.0.8] downloaded from cache at https://opam.ocaml.org/cache\n",
"\n",
"<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><> 🐫 \n",
"∗ installed conf-lapack.1\n",
"∗ installed conf-blas.1\n",
"∗ installed lacaml.11.0.8\n",
"Done.\n"
]
},
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Process.t =\n",
"{Jupyter_notebook.Process.exit_status = Unix.WEXITED 0; stdout = None;\n",
" stderr = None}\n"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Process.sh \"opam install -y lacaml\""
]
},
{
"cell_type": "markdown",
"id": "northern-public",
"metadata": {},
"source": [
"The long string syntax `{| ... |}` in OCaml is useful for writing a program of several lines."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "otherwise-wednesday",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello Alice\n"
]
},
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Process.t =\n",
"{Jupyter_notebook.Process.exit_status = Unix.WEXITED 0; stdout = None;\n",
" stderr = None}\n"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Process.python3 {|\n",
"def greeting(name: str) -> str:\n",
" return 'Hello ' + name\n",
"\n",
"print(greeting('Alice'))|}"
]
},
{
"cell_type": "markdown",
"id": "tough-zoning",
"metadata": {},
"source": [
"### 2.2. Capturing stdout/stderr\n",
"\n",
"`capture_stdout` and `capture_stderr` options controls capturing of stdout and stderr in a subprocess.\n",
"You can obtain stdout of a command in a return value by passing `~capture_stdout:true`:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "metallic-estimate",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Process.t =\n",
"{Jupyter_notebook.Process.exit_status = Unix.WEXITED 0;\n",
" stdout = Some \"This text is printed in a shell script.\\n\"; stderr = None}\n"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Process.sh ~capture_stdout:true {|echo \"This text is printed in a shell script.\"|}"
]
},
{
"cell_type": "markdown",
"id": "occasional-second",
"metadata": {},
"source": [
"``~capture_stderr:`Yes`` or ``~capture_stderr:`Redirect_to_stdout`` also captures stderr of a subprocess.\n",
"\n",
"[Process.capture_in_process](https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/Process/index.html#val-capture_in_process) captures stdout and stderr of arbitrary user functions."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "configured-karma",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"- : Jupyter_notebook.Process.t =\n",
"{Jupyter_notebook.Process.exit_status = Unix.WEXITED 0;\n",
" stdout = Some \"This text is printed in a function.\\n\"; stderr = None}\n"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Process.capture_in_process (fun () -> print_endline \"This text is printed in a function.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "strange-baseline",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "OCaml 4.13.0",
"language": "OCaml",
"name": "ocaml-jupyter-4.13.0"
},
"language_info": {
"codemirror_mode": "text/x-ocaml",
"file_extension": ".ml",
"mimetype": "text/x-ocaml",
"name": "OCaml",
"nbconverter_exporter": null,
"pygments_lexer": "OCaml",
"version": "4.13.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit 26d50e6

Please sign in to comment.