MAR_prepNetwork_03.22.2020_JB.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "last updated: 03.22.2020 by Johanna Belanger\n",
    "\n",
    "## Morocco Accessibility Model Part 1: Prep Network\n",
    "\n",
    "### Baseline model: This model measures travel time from villages in Tinghir Province, Morocco, to 3 key services of interest: health services, markets (commune centres), and schools. Travel mode is driving*. \n",
    "\n",
    "#### Walking will be modeled for villages over 5km from a road. This function is not currently in the model.\n",
    "\n",
    "Phases:\n",
    "\n",
    "- 1 | import and clean MAR network\n",
    "- 2 | snap origins and destinations to network (x3 for each facility)\n",
    "- 3 | run OD matrix to calculate driving travel time from each O to D. output is one value per each OD pair.\n",
    "- 4 | export each set of travel times to .csv and validate in QGIS\n",
    "- 5 | * Walking time model - create road features for areas over 5km from the network Graph\n",
    "- 6 | * Create Graph of walking time and add to driving Graph.\n",
    "- 7 | * Re-run steps 1-4 with new Graph object\n",
    "- 8 | * Join origins and destinations names columns to OD matrix and export output. Current functionality not available in gostnets. \n",
    "\n",
    "- *** phase has not been implemented"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# configure script\n",
    "import geopandas as gpd\n",
    "import pandas as pd\n",
    "import os, sys, time\n",
    "# set file path of GOSTnets scripts\n",
    "sys.path.append(os.path.join(r'/Users/jobelanger/GOSTnets-master/GOSTnets'))\n",
    "import GOSTnets as gn\n",
    "import networkx as nx\n",
    "import osmnx as ox\n",
    "from shapely.ops import unary_union\n",
    "from shapely.wkt import loads\n",
    "from shapely.geometry import LineString, MultiLineString, Point\n",
    "import load_osm as losm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set file path and load osm.pbf\n",
    "pth = r'/Users/jobelanger/GOSTnets-master/morocco'\n",
    "dataPth = r'/Users/jobelanger/GOSTnets-master/morocco/data'\n",
    "outPth = r'/Users/jobelanger/GOSTnets-master/morocco/outputs'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# osm pbf last downloaded on: 02.17.20\n",
    "fil = r'morocco-latest.osm.pbf'\n",
    "f = os.path.join(pth, 'data', fil)\n",
    "mar = losm.OSM_to_network(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aoi = r'/Users/jobelanger/GOSTnets-master/mar/tinghirP.shp'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shp = gpd.read_file(os.path.join(dataPth, aoi))\n",
    "shp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "shp_poly = shp.geometry.iloc[0]\n",
    "# Check that the shape looks right\n",
    "shp_poly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# create G from tinghir roads within AOI\n",
    "G = ox.graph_from_polygon(bound, network_type='drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the roads object as a pickle\n",
    "gn.save(G, 'mar_unclean', './', pickle=True, nodes=False, edges=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Clean network and export as \"clean\" networkx Graph object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set the EPSG code for Morocco (MAR).\n",
    "UTMZs = {'MAR':32629}\n",
    "\n",
    "# do not adjust. OSM natively comes in ESPG 4326.\n",
    "WGS = {'init':'epsg:4326'}\n",
    "\n",
    "# this process can clean multiple networks at once in a loop style\n",
    "countries = ['MAR']\n",
    "\n",
    "# adjust to your input filepath:\n",
    "base_pth = os.path.join(os.path.dirname(os.getcwd()), r'/Users/jobelanger/GOSTnets-master/morocco')\n",
    "data_pth = os.path.join(base_pth, 'outputs')\n",
    "\n",
    "for country in countries:\n",
    "    print('\\n--- processing for: %s ---\\n' % country)\n",
    "    print('start: %s\\n' % time.ctime())\n",
    "\n",
    "    print('Outputs can be found at: %s\\n' % (data_pth))\n",
    "        \n",
    "    UTM = {'init': 'epsg:%d' % UTMZs[country]}\n",
    "    \n",
    "    G = nx.read_gpickle(os.path.join(data_pth, 'mar_unclean.pickle'))\n",
    "    \n",
    "    G = gn.clean_network(G, data_pth, country, UTM, WGS, 0.5, verbose = False)\n",
    "    \n",
    "    nx.write_gpickle(G, os.path.join(data_pth, 'mar_clean.pickle'))\n",
    "    print('\\nend: %s' % time.ctime())\n",
    "    print('\\n--- processing complete for: %s ---' % country)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(G)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### find the largest subgraph of the network graph. \n",
    "\n",
    "network analysis will only work correctly on graphs that are fully connected, otherwise there will be error.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# find largest sub graph of your Graph.\n",
    "print('G before:')\n",
    "print(G.size())\n",
    "G_before = G.size()\n",
    "largest = max(nx.strongly_connected_components(G), key=len)\n",
    "G = nx.induced_subgraph(G, largest).copy()\n",
    "print('G after:')\n",
    "print(G.size())\n",
    "G_after = G.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# calculate the percent of the network that is completed.\n",
    "# analysis will only work correctly on graphs that are connected.\n",
    "G_diff = (G_before - G_after)\n",
    "G_diffPerc = (G_diff/G_before) * 100\n",
    "G_percent = 100 - G_diffPerc\n",
    "print(G_diff)\n",
    "print(G_diffPerc)\n",
    "print(G_percent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the largest subgraph\n",
    "gn.save(G, 'G_largest', './', pickle=True, nodes=False, edges=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### convert your network graph (default measurement in length) to a graph mesured in time.\n",
    "\n",
    "uses the gostnets convert_network_to_time function and takes a speed dictionary and road colmn attribute as inputs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "speedDict = {\n",
    "                'residential': 30,  # kmph\n",
    "                'primary': 60, # kmph\n",
    "                'primary_link':55,\n",
    "                'trunk': 40,\n",
    "                'trunk_link':35,\n",
    "                'secondary': 50, # kmph\n",
    "                'secondary_link':45,\n",
    "                'tertiary':40,\n",
    "                'tertiary_link': 35,\n",
    "                'unclassified':30, \n",
    "                'road':20,\n",
    "                'crossing':20,\n",
    "                'living_street':10\n",
    "    \n",
    "                } "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert network to time in minutes. use factor of 1000 to convert from km to meters\n",
    "G_time = gn.convert_network_to_time(G, \n",
    "                                    distance_tag = 'length', \n",
    "                                    road_col = 'infra_type', \n",
    "                                    speed_dict = speedDict, \n",
    "                                    factor = 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save G_time object as pickle\n",
    "gn.save(G_time, 'G_time', './', pickle=True, nodes=False, edges=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### now move on to the next script to run the OD matrices: \"MAR_OD_03.22.2020_JB\""
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}