Update doc: how to development a new environment (metadriverse#631)

* update scenario description * add new environment develop * development * make better execution queue * add doc * fix a potential bug * automatically generate colab links * doc * move top down renderer to engine * finish a part * fix bug * add * allow reset to return info * Done * format * step return * wrap info * add map section * fix bug
commaai · Feb 6, 2024 · 637a221 · 637a221
1 parent 01e7f89
commit 637a221
Show file tree

Hide file tree

Showing 42 changed files with 931 additions and 386 deletions.
diff --git a/.gitignore b/.gitignore
@@ -51,3 +51,5 @@
 **/**.mp4
 /documentation/jupyter_execute/
 /documentation/**/**.gif
+/documentation/source/img.png
+/documentation/source/demo.png
diff --git a/documentation/README.md b/documentation/README.md
@@ -79,4 +79,8 @@ And you need to skip all cells you don't want to run with cell tags:
 
 ```python
 {"tags": ["skip-execution"]}
-```
+```
+
+### Preprocess-Add colab link
+Everytime the `make html` is called, all `ipynb` file will have corresponding colab link in it by modifying the source
+code. So you will find that your source code is modified in git. Please commit that changes as well.
diff --git a/documentation/source/action.ipynb b/documentation/source/action.ipynb
@@ -6,6 +6,9 @@
    "metadata": {},
    "source": [
     "# Action and Policy\n",
+    "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/action.ipynb)\n",
+    "\n",
     "This section will discuss how to control the vehicle in MetaDrive with the *Policy* interface. Before this, let's have a look at the raw control signal required by vehicles.\n",
     "\n",
     "## Action\n",
@@ -786,7 +789,7 @@
    "metadata": {},
    "source": [
     "### ReplayPolicy\n",
-    "Usually, we use [ScenarioNet](https://metadriverse.github.io/scenarionet/) to convert scenarios collected by Waymo, nuScenes, nuPlan, and Argoverse to internal [Scenario Description](description.ipynb) and rebuild the scenarios in MetaDrive. Thus this policy is used to load and replay the recorded traffic trajectories for these real-world scenarios. There are two replay policies for replaying the trajectories of ego car and traffic participants, respectively:\n",
+    "Usually, we use [ScenarioNet](https://metadriverse.github.io/scenarionet/) to convert scenarios collected by Waymo, nuScenes, nuPlan, and Argoverse to internal [Scenario Description](scenario_description.ipynb) and rebuild the scenarios in MetaDrive. Thus this policy is used to load and replay the recorded traffic trajectories for these real-world scenarios. There are two replay policies for replaying the trajectories of ego car and traffic participants, respectively:\n",
     "* **ReplayEgoCarPolicy**\n",
     "* **ReplayTrafficParticipantPolicy**\n",
     "\n",

diff --git a/documentation/source/before_reading.ipynb b/documentation/source/before_reading.ipynb
@@ -6,6 +6,9 @@
    "metadata": {},
    "source": [
     "# Before reading the Doc\n",
+    "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/before_reading.ipynb)\n",
+    "\n",
     "Most of the document is made with `jupyter notebook`. Therefore, you can clone the [repo](https://github.com/metadriverse/metadrive) and run the demo code listed in `metadrive/documentation/source/*.ipynb` on your local machine. These examples are supposed to work with the latest main branch. Note: videos can not be played in notebook as it doesn't support `<video>` label.\n",
     "\n",
     "Some code cell contains the expected running results. If you find your running is different from the existing cell output or the code is broken, please file an issue in our [github repo](https://github.com/metadriverse/metadrive). "

diff --git a/documentation/source/conf.py b/documentation/source/conf.py
@@ -74,3 +74,39 @@
     "html_image",
 ]
 myst_url_schemes = ("http", "https", "mailto")
+
+# This will add the colab execution link automatically!
+import os
+import nbformat
+
+
+def add_hyperlink_to_notebook(notebook_path):
+    """
+    Add the colab link to notebook!
+    """
+    nb = nbformat.read(notebook_path, as_version=4)
+    link_text = "https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/{}".format(
+        os.path.basename(notebook_path))
+    link_text = "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]({})".format(
+        link_text)
+    # Assuming you want to add the link below the first heading
+    cell = nb.cells[0]
+    assert cell.cell_type == 'markdown', cell
+    if "colab" not in cell.source:
+        p = cell.source.find("\n")
+        cell.source = cell.source[:p] + f"\n\n{link_text}\n" + cell.source[p:]
+        # print(cell.source)
+        nbformat.write(nb, notebook_path)
+
+
+def process_notebooks(directory):
+    """
+    Add to all notebook.
+    """
+    for root, dirs, files in os.walk(directory):
+        for file in files:
+            if file.endswith(".ipynb"):
+                add_hyperlink_to_notebook(os.path.join(root, file))
+
+
+process_notebooks(os.path.dirname(__file__))
diff --git a/documentation/source/config_system.ipynb b/documentation/source/config_system.ipynb
@@ -7,6 +7,9 @@
    "source": [
     "# Configuration\n",
     "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/config_system.ipynb)\n",
+    "\n",
+    "\n",
     "A MetaDrive instance accepts a dict as the environmental config. For example, you can build a MetaDrive instance with 200 generated maps via \n",
     "```python\n",
     "from metadrive import MetaDriveEnv\n",

diff --git a/documentation/source/debug_mode.ipynb b/documentation/source/debug_mode.ipynb
@@ -6,6 +6,9 @@
    "metadata": {},
    "source": [
     "# Debugging mode\n",
+    "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/debug_mode.ipynb)\n",
+    "\n",
     "Some configurations can be turned on to debug the simulator. As a result, you can get more console output from the simulation and visualizations such as coordinates and physics to help develop features or fix bugs. To receive debug messages, set `debug=True` in the config when creating the environment. In debug mode, the `log_level` will be changed to `logging.DEBUG`, so you will get more information from MetaDrive, Panda3D, and RenderPipeline in the console."
    ]
   },

diff --git a/documentation/source/development.rst b/documentation/source/development.rst
diff --git a/documentation/source/figs/execution_queue.png b/documentation/source/figs/execution_queue.png
diff --git a/documentation/source/index.rst b/documentation/source/index.rst
@@ -53,15 +53,16 @@ Please feel free to contact us if you have any suggestions or ideas!
    :maxdepth: 2
    :caption: Concepts and Customization
 
+   new_env.ipynb
    system_design.ipynb
    sensors.ipynb
    top_down_render.ipynb
    panda_render.ipynb
+   map.ipynb
    vehicle.ipynb
    navigation.ipynb
    scenario_description.ipynb
    record_replay.ipynb
-   development.rst
 
 .. toctree::
    :hidden:

diff --git a/documentation/source/log_msg.ipynb b/documentation/source/log_msg.ipynb
@@ -7,6 +7,9 @@
    "source": [
     "# Log messages\n",
     "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/log_msg.ipynb)\n",
+    "\n",
+    "\n",
     "Logging message is important to oversee the program status. MetaDrive adopts Python's `logging` module to log messages. "
    ]
   },

diff --git a/documentation/source/map.ipynb b/documentation/source/map.ipynb
@@ -0,0 +1,42 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8b81ba11-b84b-4acd-85ee-e41d71b379b4",
+   "metadata": {},
+   "source": [
+    "# Map\n",
+    "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/map.ipynb)\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## PGMap\n",
+    "\n",
+    "## ScenarioMap"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.17"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/documentation/source/navigation.ipynb b/documentation/source/navigation.ipynb
@@ -5,7 +5,9 @@
    "id": "008aaa7c",
    "metadata": {},
    "source": [
-    "# Navigation"
+    "# Navigation\n",
+    "\n",
+    "[![Click and Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/metadriverse/metadrive/blob/main/documentation/source/navigation.ipynb)"
    ]
   }
  ],
@@ -25,7 +27,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.13"
+   "version": "3.9.17"
   }
  },
  "nbformat": 4,