DP0701EN-2-2-1-Foursquare-API-py-v1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://cognitiveclass.ai\"><img src = \"https://ibm.box.com/shared/static/9gegpsmnsoo25ikkbl4qzlvlyjbgxs5x.png\" width = 400> </a>\n",
    "\n",
    "<h1 align=center><font size = 5>Learning FourSquare API with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Introduction\n",
    "\n",
    "In this lab, you will learn in details how to make calls to the Foursquare API for different purposes. You will learn how to construct a URL to send a request to the API to search for a specific type of venues, to explore a particular venue, to explore a Foursquare user, to explore a geographical location, and to get trending venues around a location. Also, you will learn how to use the visualization library, Folium, to visualize the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Table of Contents\n",
    "\n",
    "1. <a href=\"#item1\">Foursquare API Search Function</a>\n",
    "2. <a href=\"#item2\">Explore a Given Venue</a>  \n",
    "3. <a href=\"#item3\">Explore a User</a>  \n",
    "4. <a href=\"#item4\">Foursquare API Explore Function</a>  \n",
    "5. <a href=\"#item5\">Get Trending Venues</a>  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Import necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting geopy\n",
      "  Downloading geopy-1.21.0-py2.py3-none-any.whl (104 kB)\n",
      "Collecting geographiclib<2,>=1.49\n",
      "  Downloading geographiclib-1.50-py3-none-any.whl (38 kB)\n",
      "Installing collected packages: geographiclib, geopy\n",
      "Successfully installed geographiclib-1.50 geopy-1.21.0\n",
      "Requirement already satisfied: folium in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (0.10.1)\n",
      "Requirement already satisfied: numpy in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from folium) (1.18.1)\n",
      "Requirement already satisfied: requests in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from folium) (2.23.0)\n",
      "Requirement already satisfied: branca>=0.3.0 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from folium) (0.4.0)\n",
      "Requirement already satisfied: jinja2>=2.9 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from folium) (2.11.1)\n",
      "Requirement already satisfied: idna<3,>=2.5 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from requests->folium) (2.9)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from requests->folium) (1.25.8)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from requests->folium) (2019.11.28)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from requests->folium) (3.0.4)\n",
      "Requirement already satisfied: six in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from branca>=0.3.0->folium) (1.14.0)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in c:\\users\\nguyenthevinh\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages (from jinja2>=2.9->folium) (1.1.1)\n"
     ]
    }
   ],
   "source": [
    "!pip install geopy\n",
    "!pip install folium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Folium installed\n",
      "Libraries imported.\n"
     ]
    }
   ],
   "source": [
    "import requests # library to handle requests\n",
    "import pandas as pd # library for data analsysis\n",
    "import numpy as np # library to handle data in a vectorized manner\n",
    "import random # library for random number generation\n",
    " \n",
    "from geopy.geocoders import Nominatim # module to convert an address into latitude and longitude values\n",
    "\n",
    "# libraries for displaying images\n",
    "from IPython.display import Image \n",
    "from IPython.core.display import HTML \n",
    "    \n",
    "# tranforming json file into a pandas dataframe library\n",
    "from pandas.io.json import json_normalize\n",
    "\n",
    "import folium # plotting library\n",
    "\n",
    "print('Folium installed')\n",
    "print('Libraries imported.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Define Foursquare Credentials and Version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "##### Make sure that you have created a Foursquare developer account and have your credentials handy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Your credentails:\n",
      "CLIENT_ID: H0IZ23LNW3Y4DDUGZEPW4SBIVS2QCQE0SKLAMP15UZMQGPPU\n",
      "CLIENT_SECRET:IUSOYUQKPVUKYOYYP4OV1WB1BPOJ4IGI0R0JBS4UPDPEIYFS\n"
     ]
    }
   ],
   "source": [
    "CLIENT_ID = 'H0IZ23LNW3Y4DDUGZEPW4SBIVS2QCQE0SKLAMP15UZMQGPPU' # your Foursquare ID\n",
    "CLIENT_SECRET = 'IUSOYUQKPVUKYOYYP4OV1WB1BPOJ4IGI0R0JBS4UPDPEIYFS' # your Foursquare Secret\n",
    "VERSION = '20201103'\n",
    "LIMIT = 30\n",
    "print('Your credentails:')\n",
    "print('CLIENT_ID: ' + CLIENT_ID)\n",
    "print('CLIENT_SECRET:' + CLIENT_SECRET)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Let's again assume that you are staying at the Conrad hotel. So let's start by converting the Contrad Hotel's address to its latitude and longitude coordinates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to define an instance of the geocoder, we need to define a user_agent. We will name our agent <em>foursquare_agent</em>, as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "40.7149555 -74.0153365\n"
     ]
    }
   ],
   "source": [
    "address = '102 North End Ave, New York, NY'\n",
    "\n",
    "geolocator = Nominatim(user_agent=\"foursquare_agent\")\n",
    "location = geolocator.geocode(address)\n",
    "latitude = location.latitude\n",
    "longitude = location.longitude\n",
    "print(latitude, longitude)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## 1. Search for a specific venue category\n",
    "> `https://api.foursquare.com/v2/venues/`**search**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**`&query=`**QUERY**`&radius=`**RADIUS**`&limit=`**LIMIT**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Now, let's assume that it is lunch time, and you are craving Italian food. So, let's define a query to search for Italian food that is within 500 metres from the Conrad Hotel. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Italian .... OK!\n"
     ]
    }
   ],
   "source": [
    "search_query = 'Italian'\n",
    "radius = 1000\n",
    "print(search_query + ' .... OK!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Define the corresponding URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://api.foursquare.com/v2/venues/search?client_id=H0IZ23LNW3Y4DDUGZEPW4SBIVS2QCQE0SKLAMP15UZMQGPPU&client_secret=IUSOYUQKPVUKYOYYP4OV1WB1BPOJ4IGI0R0JBS4UPDPEIYFS&ll=40.7149555,-74.0153365&v=20201103&query=Italian&radius=1000&limit=30'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = 'https://api.foursquare.com/v2/venues/search?client_id={}&client_secret={}&ll={},{}&v={}&query={}&radius={}&limit={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION, search_query, radius, LIMIT)\n",
    "url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Send the GET Request and examine the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'meta': {'code': 200, 'requestId': '5e68734002a172001b901133'},\n",
       " 'response': {'venues': [{'id': '4a4560a7f964a5201aa81fe3',\n",
       "    'name': \"Harry's Italian Pizza Bar\",\n",
       "    'location': {'address': '2 Gold St',\n",
       "     'crossStreet': 'at Maiden Ln.',\n",
       "     'lat': 40.7076335455641,\n",
       "     'lng': -74.00699585960496,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.7076335455641,\n",
       "       'lng': -74.00699585960496}],\n",
       "     'distance': 1076,\n",
       "     'postalCode': '10038',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['2 Gold St (at Maiden Ln.)',\n",
       "      'New York, NY 10038',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d110941735',\n",
       "      'name': 'Italian Restaurant',\n",
       "      'pluralName': 'Italian Restaurants',\n",
       "      'shortName': 'Italian',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'delivery': {'id': '294543',\n",
       "     'url': 'https://www.seamless.com/menu/harrys-italian-2-gold-st-new-york/294543?affiliate=1131&utm_source=foursquare-affiliate-network&utm_medium=affiliate&utm_campaign=1131&utm_content=294543',\n",
       "     'provider': {'name': 'seamless',\n",
       "      'icon': {'prefix': 'https://fastly.4sqi.net/img/general/cap/',\n",
       "       'sizes': [40, 50],\n",
       "       'name': '/delivery_provider_seamless_20180129.png'}}},\n",
       "    'venuePage': {'id': '77559825'},\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4fa862b3e4b0ebff2f749f06',\n",
       "    'name': \"Harry's Italian Pizza Bar\",\n",
       "    'location': {'address': '225 Murray St',\n",
       "     'lat': 40.71521779064671,\n",
       "     'lng': -74.01473940209351,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.71521779064671,\n",
       "       'lng': -74.01473940209351}],\n",
       "     'distance': 58,\n",
       "     'postalCode': '10282',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['225 Murray St',\n",
       "      'New York, NY 10282',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d1ca941735',\n",
       "      'name': 'Pizza Place',\n",
       "      'pluralName': 'Pizza Places',\n",
       "      'shortName': 'Pizza',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/pizza_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'delivery': {'id': '294544',\n",
       "     'url': 'https://www.seamless.com/menu/harrys-italian-pizza-bar-225-murray-st-new-york/294544?affiliate=1131&utm_source=foursquare-affiliate-network&utm_medium=affiliate&utm_campaign=1131&utm_content=294544',\n",
       "     'provider': {'name': 'seamless',\n",
       "      'icon': {'prefix': 'https://fastly.4sqi.net/img/general/cap/',\n",
       "       'sizes': [40, 50],\n",
       "       'name': '/delivery_provider_seamless_20180129.png'}}},\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '528bf16711d2b7722da6b51c',\n",
       "    'name': 'Pisillo Italian Panini',\n",
       "    'location': {'address': '97 Nassau St',\n",
       "     'crossStreet': 'at Ann St',\n",
       "     'lat': 40.710530084460785,\n",
       "     'lng': -74.00752571804911,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.710530084460785,\n",
       "       'lng': -74.00752571804911}],\n",
       "     'distance': 822,\n",
       "     'postalCode': '10038',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['97 Nassau St (at Ann St)',\n",
       "      'New York, NY 10038',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d1c5941735',\n",
       "      'name': 'Sandwich Place',\n",
       "      'pluralName': 'Sandwich Places',\n",
       "      'shortName': 'Sandwiches',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/deli_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'delivery': {'id': '286355',\n",
       "     'url': 'https://www.seamless.com/menu/pisillo-97-nassau-st-new-york/286355?affiliate=1131&utm_source=foursquare-affiliate-network&utm_medium=affiliate&utm_campaign=1131&utm_content=286355',\n",
       "     'provider': {'name': 'seamless',\n",
       "      'icon': {'prefix': 'https://fastly.4sqi.net/img/general/cap/',\n",
       "       'sizes': [40, 50],\n",
       "       'name': '/delivery_provider_seamless_20180129.png'}}},\n",
       "    'venuePage': {'id': '105457747'},\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4f3feae2e4b0ac9416a72118',\n",
       "    'name': 'Benvenuti Italian Specialties',\n",
       "    'location': {'address': '235 S End Ave',\n",
       "     'lat': 40.70968105555279,\n",
       "     'lng': -74.01652043988257,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.70968105555279,\n",
       "       'lng': -74.01652043988257}],\n",
       "     'distance': 595,\n",
       "     'postalCode': '10280',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['235 S End Ave',\n",
       "      'New York, NY 10280',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d1f5941735',\n",
       "      'name': 'Gourmet Shop',\n",
       "      'pluralName': 'Gourmet Shops',\n",
       "      'shortName': 'Gourmet',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/shops/food_gourmet_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'delivery': {'id': '285069',\n",
       "     'url': 'https://www.seamless.com/menu/benvenuti-235-s-end-ave-new-york/285069?affiliate=1131&utm_source=foursquare-affiliate-network&utm_medium=affiliate&utm_campaign=1131&utm_content=285069',\n",
       "     'provider': {'name': 'seamless',\n",
       "      'icon': {'prefix': 'https://fastly.4sqi.net/img/general/cap/',\n",
       "       'sizes': [40, 50],\n",
       "       'name': '/delivery_provider_seamless_20180129.png'}}},\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4f3232e219836c91c7bfde94',\n",
       "    'name': 'Conca Cucina Italian Restaurant',\n",
       "    'location': {'address': '63 W Broadway',\n",
       "     'lat': 40.714484000000006,\n",
       "     'lng': -74.00980600000001,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.714484000000006,\n",
       "       'lng': -74.00980600000001}],\n",
       "     'distance': 469,\n",
       "     'postalCode': '10007',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['63 W Broadway',\n",
       "      'New York, NY 10007',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4d4b7105d754a06374d81259',\n",
       "      'name': 'Food',\n",
       "      'pluralName': 'Food',\n",
       "      'shortName': 'Food',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/default_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '588a4316326c5a4b60559f17',\n",
       "    'name': 'Pisillo Italian Cafe',\n",
       "    'location': {'lat': 40.71049280792626,\n",
       "     'lng': -74.00754611675153,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.71049280792626,\n",
       "       'lng': -74.00754611675153}],\n",
       "     'distance': 823,\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['New York, NY', 'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d16d941735',\n",
       "      'name': 'Café',\n",
       "      'pluralName': 'Cafés',\n",
       "      'shortName': 'Café',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/cafe_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4e4c4d28bd413c4cc66857a5',\n",
       "    'name': \"Caruso's Italian Restaurant\",\n",
       "    'location': {'address': '42 Broadway',\n",
       "     'lat': 40.710509099999996,\n",
       "     'lng': -74.00827615,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.710509099999996,\n",
       "       'lng': -74.00827615}],\n",
       "     'distance': 774,\n",
       "     'postalCode': '10004',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['42 Broadway',\n",
       "      'New York, NY 10004',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d110941735',\n",
       "      'name': 'Italian Restaurant',\n",
       "      'pluralName': 'Italian Restaurants',\n",
       "      'shortName': 'Italian',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4c3b8244def90f478f0ffe2d',\n",
       "    'name': 'Barber Italian Shop',\n",
       "    'location': {'address': '120 Broadway',\n",
       "     'crossStreet': 'btw Cedar & Pine',\n",
       "     'lat': 40.70796968555732,\n",
       "     'lng': -74.00994920143108,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.70796968555732,\n",
       "       'lng': -74.00994920143108}],\n",
       "     'distance': 900,\n",
       "     'postalCode': '10271',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['120 Broadway (btw Cedar & Pine)',\n",
       "      'New York, NY 10271',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d103951735',\n",
       "      'name': 'Clothing Store',\n",
       "      'pluralName': 'Clothing Stores',\n",
       "      'shortName': 'Apparel',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/shops/apparel_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '3fd66200f964a520f4e41ee3',\n",
       "    'name': 'Ecco',\n",
       "    'location': {'address': '124 Chambers St',\n",
       "     'crossStreet': 'btwn Church St & W Broadway',\n",
       "     'lat': 40.71533713859952,\n",
       "     'lng': -74.00884766217825,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.71533713859952,\n",
       "       'lng': -74.00884766217825}],\n",
       "     'distance': 549,\n",
       "     'postalCode': '10007',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['124 Chambers St (btwn Church St & W Broadway)',\n",
       "      'New York, NY 10007',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d110941735',\n",
       "      'name': 'Italian Restaurant',\n",
       "      'pluralName': 'Italian Restaurants',\n",
       "      'shortName': 'Italian',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False},\n",
       "   {'id': '4c5de89e6ebe2d7f7955d42e',\n",
       "    'name': \"Gloria's Restaurant\",\n",
       "    'location': {'address': '107 W Broadway',\n",
       "     'crossStreet': 'Reade St',\n",
       "     'lat': 40.710412,\n",
       "     'lng': -74.007998,\n",
       "     'labeledLatLngs': [{'label': 'display',\n",
       "       'lat': 40.710412,\n",
       "       'lng': -74.007998}],\n",
       "     'distance': 799,\n",
       "     'postalCode': '10013',\n",
       "     'cc': 'US',\n",
       "     'city': 'New York',\n",
       "     'state': 'NY',\n",
       "     'country': 'United States',\n",
       "     'formattedAddress': ['107 W Broadway (Reade St)',\n",
       "      'New York, NY 10013',\n",
       "      'United States']},\n",
       "    'categories': [{'id': '4bf58dd8d48988d110941735',\n",
       "      'name': 'Italian Restaurant',\n",
       "      'pluralName': 'Italian Restaurants',\n",
       "      'shortName': 'Italian',\n",
       "      'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n",
       "       'suffix': '.png'},\n",
       "      'primary': True}],\n",
       "    'referralId': 'v-1583903991',\n",
       "    'hasPerk': False}]}}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = requests.get(url).json()\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Get relevant part of JSON and transform it into a *pandas* dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-19-5acf500bf9ad>:5: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n",
      "  dataframe = json_normalize(venues)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>categories</th>\n",
       "      <th>referralId</th>\n",
       "      <th>hasPerk</th>\n",
       "      <th>location.address</th>\n",
       "      <th>location.crossStreet</th>\n",
       "      <th>location.lat</th>\n",
       "      <th>location.lng</th>\n",
       "      <th>location.labeledLatLngs</th>\n",
       "      <th>...</th>\n",
       "      <th>location.state</th>\n",
       "      <th>location.country</th>\n",
       "      <th>location.formattedAddress</th>\n",
       "      <th>delivery.id</th>\n",
       "      <th>delivery.url</th>\n",
       "      <th>delivery.provider.name</th>\n",
       "      <th>delivery.provider.icon.prefix</th>\n",
       "      <th>delivery.provider.icon.sizes</th>\n",
       "      <th>delivery.provider.icon.name</th>\n",
       "      <th>venuePage.id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4a4560a7f964a5201aa81fe3</td>\n",
       "      <td>Harry's Italian Pizza Bar</td>\n",
       "      <td>[{'id': '4bf58dd8d48988d110941735', 'name': 'I...</td>\n",
       "      <td>v-1583903991</td>\n",
       "      <td>False</td>\n",
       "      <td>2 Gold St</td>\n",
       "      <td>at Maiden Ln.</td>\n",
       "      <td>40.707634</td>\n",
       "      <td>-74.006996</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.7076335455641,...</td>\n",
       "      <td>...</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[2 Gold St (at Maiden Ln.), New York, NY 10038...</td>\n",
       "      <td>294543</td>\n",
       "      <td>https://www.seamless.com/menu/harrys-italian-2...</td>\n",
       "      <td>seamless</td>\n",
       "      <td>https://fastly.4sqi.net/img/general/cap/</td>\n",
       "      <td>[40, 50]</td>\n",
       "      <td>/delivery_provider_seamless_20180129.png</td>\n",
       "      <td>77559825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4fa862b3e4b0ebff2f749f06</td>\n",
       "      <td>Harry's Italian Pizza Bar</td>\n",
       "      <td>[{'id': '4bf58dd8d48988d1ca941735', 'name': 'P...</td>\n",
       "      <td>v-1583903991</td>\n",
       "      <td>False</td>\n",
       "      <td>225 Murray St</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.715218</td>\n",
       "      <td>-74.014739</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71521779064671...</td>\n",
       "      <td>...</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[225 Murray St, New York, NY 10282, United Sta...</td>\n",
       "      <td>294544</td>\n",
       "      <td>https://www.seamless.com/menu/harrys-italian-p...</td>\n",
       "      <td>seamless</td>\n",
       "      <td>https://fastly.4sqi.net/img/general/cap/</td>\n",
       "      <td>[40, 50]</td>\n",
       "      <td>/delivery_provider_seamless_20180129.png</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>528bf16711d2b7722da6b51c</td>\n",
       "      <td>Pisillo Italian Panini</td>\n",
       "      <td>[{'id': '4bf58dd8d48988d1c5941735', 'name': 'S...</td>\n",
       "      <td>v-1583903991</td>\n",
       "      <td>False</td>\n",
       "      <td>97 Nassau St</td>\n",
       "      <td>at Ann St</td>\n",
       "      <td>40.710530</td>\n",
       "      <td>-74.007526</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71053008446078...</td>\n",
       "      <td>...</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[97 Nassau St (at Ann St), New York, NY 10038,...</td>\n",
       "      <td>286355</td>\n",
       "      <td>https://www.seamless.com/menu/pisillo-97-nassa...</td>\n",
       "      <td>seamless</td>\n",
       "      <td>https://fastly.4sqi.net/img/general/cap/</td>\n",
       "      <td>[40, 50]</td>\n",
       "      <td>/delivery_provider_seamless_20180129.png</td>\n",
       "      <td>105457747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4f3feae2e4b0ac9416a72118</td>\n",
       "      <td>Benvenuti Italian Specialties</td>\n",
       "      <td>[{'id': '4bf58dd8d48988d1f5941735', 'name': 'G...</td>\n",
       "      <td>v-1583903991</td>\n",
       "      <td>False</td>\n",
       "      <td>235 S End Ave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.709681</td>\n",
       "      <td>-74.016520</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.70968105555279...</td>\n",
       "      <td>...</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[235 S End Ave, New York, NY 10280, United Sta...</td>\n",
       "      <td>285069</td>\n",
       "      <td>https://www.seamless.com/menu/benvenuti-235-s-...</td>\n",
       "      <td>seamless</td>\n",
       "      <td>https://fastly.4sqi.net/img/general/cap/</td>\n",
       "      <td>[40, 50]</td>\n",
       "      <td>/delivery_provider_seamless_20180129.png</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4f3232e219836c91c7bfde94</td>\n",
       "      <td>Conca Cucina Italian Restaurant</td>\n",
       "      <td>[{'id': '4d4b7105d754a06374d81259', 'name': 'F...</td>\n",
       "      <td>v-1583903991</td>\n",
       "      <td>False</td>\n",
       "      <td>63 W Broadway</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.714484</td>\n",
       "      <td>-74.009806</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71448400000000...</td>\n",
       "      <td>...</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[63 W Broadway, New York, NY 10007, United Sta...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         id                             name  \\\n",
       "0  4a4560a7f964a5201aa81fe3        Harry's Italian Pizza Bar   \n",
       "1  4fa862b3e4b0ebff2f749f06        Harry's Italian Pizza Bar   \n",
       "2  528bf16711d2b7722da6b51c           Pisillo Italian Panini   \n",
       "3  4f3feae2e4b0ac9416a72118    Benvenuti Italian Specialties   \n",
       "4  4f3232e219836c91c7bfde94  Conca Cucina Italian Restaurant   \n",
       "\n",
       "                                          categories    referralId  hasPerk  \\\n",
       "0  [{'id': '4bf58dd8d48988d110941735', 'name': 'I...  v-1583903991    False   \n",
       "1  [{'id': '4bf58dd8d48988d1ca941735', 'name': 'P...  v-1583903991    False   \n",
       "2  [{'id': '4bf58dd8d48988d1c5941735', 'name': 'S...  v-1583903991    False   \n",
       "3  [{'id': '4bf58dd8d48988d1f5941735', 'name': 'G...  v-1583903991    False   \n",
       "4  [{'id': '4d4b7105d754a06374d81259', 'name': 'F...  v-1583903991    False   \n",
       "\n",
       "  location.address location.crossStreet  location.lat  location.lng  \\\n",
       "0        2 Gold St        at Maiden Ln.     40.707634    -74.006996   \n",
       "1    225 Murray St                  NaN     40.715218    -74.014739   \n",
       "2     97 Nassau St            at Ann St     40.710530    -74.007526   \n",
       "3    235 S End Ave                  NaN     40.709681    -74.016520   \n",
       "4    63 W Broadway                  NaN     40.714484    -74.009806   \n",
       "\n",
       "                             location.labeledLatLngs  ...  location.state  \\\n",
       "0  [{'label': 'display', 'lat': 40.7076335455641,...  ...              NY   \n",
       "1  [{'label': 'display', 'lat': 40.71521779064671...  ...              NY   \n",
       "2  [{'label': 'display', 'lat': 40.71053008446078...  ...              NY   \n",
       "3  [{'label': 'display', 'lat': 40.70968105555279...  ...              NY   \n",
       "4  [{'label': 'display', 'lat': 40.71448400000000...  ...              NY   \n",
       "\n",
       "  location.country                          location.formattedAddress  \\\n",
       "0    United States  [2 Gold St (at Maiden Ln.), New York, NY 10038...   \n",
       "1    United States  [225 Murray St, New York, NY 10282, United Sta...   \n",
       "2    United States  [97 Nassau St (at Ann St), New York, NY 10038,...   \n",
       "3    United States  [235 S End Ave, New York, NY 10280, United Sta...   \n",
       "4    United States  [63 W Broadway, New York, NY 10007, United Sta...   \n",
       "\n",
       "  delivery.id                                       delivery.url  \\\n",
       "0      294543  https://www.seamless.com/menu/harrys-italian-2...   \n",
       "1      294544  https://www.seamless.com/menu/harrys-italian-p...   \n",
       "2      286355  https://www.seamless.com/menu/pisillo-97-nassa...   \n",
       "3      285069  https://www.seamless.com/menu/benvenuti-235-s-...   \n",
       "4         NaN                                                NaN   \n",
       "\n",
       "  delivery.provider.name             delivery.provider.icon.prefix  \\\n",
       "0               seamless  https://fastly.4sqi.net/img/general/cap/   \n",
       "1               seamless  https://fastly.4sqi.net/img/general/cap/   \n",
       "2               seamless  https://fastly.4sqi.net/img/general/cap/   \n",
       "3               seamless  https://fastly.4sqi.net/img/general/cap/   \n",
       "4                    NaN                                       NaN   \n",
       "\n",
       "  delivery.provider.icon.sizes               delivery.provider.icon.name  \\\n",
       "0                     [40, 50]  /delivery_provider_seamless_20180129.png   \n",
       "1                     [40, 50]  /delivery_provider_seamless_20180129.png   \n",
       "2                     [40, 50]  /delivery_provider_seamless_20180129.png   \n",
       "3                     [40, 50]  /delivery_provider_seamless_20180129.png   \n",
       "4                          NaN                                       NaN   \n",
       "\n",
       "  venuePage.id  \n",
       "0     77559825  \n",
       "1          NaN  \n",
       "2    105457747  \n",
       "3          NaN  \n",
       "4          NaN  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# assign relevant part of JSON to venues\n",
    "venues = results['response']['venues']\n",
    "\n",
    "# tranform venues into a dataframe\n",
    "dataframe = json_normalize(venues)\n",
    "dataframe.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Define information of interest and filter dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>categories</th>\n",
       "      <th>address</th>\n",
       "      <th>crossStreet</th>\n",
       "      <th>lat</th>\n",
       "      <th>lng</th>\n",
       "      <th>labeledLatLngs</th>\n",
       "      <th>distance</th>\n",
       "      <th>postalCode</th>\n",
       "      <th>cc</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>country</th>\n",
       "      <th>formattedAddress</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Harry's Italian Pizza Bar</td>\n",
       "      <td>Italian Restaurant</td>\n",
       "      <td>2 Gold St</td>\n",
       "      <td>at Maiden Ln.</td>\n",
       "      <td>40.707634</td>\n",
       "      <td>-74.006996</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.7076335455641, 'lng': -74.00699585960496}]</td>\n",
       "      <td>1076</td>\n",
       "      <td>10038</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[2 Gold St (at Maiden Ln.), New York, NY 10038, United States]</td>\n",
       "      <td>4a4560a7f964a5201aa81fe3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Harry's Italian Pizza Bar</td>\n",
       "      <td>Pizza Place</td>\n",
       "      <td>225 Murray St</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.715218</td>\n",
       "      <td>-74.014739</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71521779064671, 'lng': -74.01473940209351}]</td>\n",
       "      <td>58</td>\n",
       "      <td>10282</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[225 Murray St, New York, NY 10282, United States]</td>\n",
       "      <td>4fa862b3e4b0ebff2f749f06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Pisillo Italian Panini</td>\n",
       "      <td>Sandwich Place</td>\n",
       "      <td>97 Nassau St</td>\n",
       "      <td>at Ann St</td>\n",
       "      <td>40.710530</td>\n",
       "      <td>-74.007526</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.710530084460785, 'lng': -74.00752571804911}]</td>\n",
       "      <td>822</td>\n",
       "      <td>10038</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[97 Nassau St (at Ann St), New York, NY 10038, United States]</td>\n",
       "      <td>528bf16711d2b7722da6b51c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Benvenuti Italian Specialties</td>\n",
       "      <td>Gourmet Shop</td>\n",
       "      <td>235 S End Ave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.709681</td>\n",
       "      <td>-74.016520</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.70968105555279, 'lng': -74.01652043988257}]</td>\n",
       "      <td>595</td>\n",
       "      <td>10280</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[235 S End Ave, New York, NY 10280, United States]</td>\n",
       "      <td>4f3feae2e4b0ac9416a72118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Conca Cucina Italian Restaurant</td>\n",
       "      <td>Food</td>\n",
       "      <td>63 W Broadway</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.714484</td>\n",
       "      <td>-74.009806</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.714484000000006, 'lng': -74.00980600000001}]</td>\n",
       "      <td>469</td>\n",
       "      <td>10007</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[63 W Broadway, New York, NY 10007, United States]</td>\n",
       "      <td>4f3232e219836c91c7bfde94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Pisillo Italian Cafe</td>\n",
       "      <td>Café</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.710493</td>\n",
       "      <td>-74.007546</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71049280792626, 'lng': -74.00754611675153}]</td>\n",
       "      <td>823</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[New York, NY, United States]</td>\n",
       "      <td>588a4316326c5a4b60559f17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Caruso's Italian Restaurant</td>\n",
       "      <td>Italian Restaurant</td>\n",
       "      <td>42 Broadway</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.710509</td>\n",
       "      <td>-74.008276</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.710509099999996, 'lng': -74.00827615}]</td>\n",
       "      <td>774</td>\n",
       "      <td>10004</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[42 Broadway, New York, NY 10004, United States]</td>\n",
       "      <td>4e4c4d28bd413c4cc66857a5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Barber Italian Shop</td>\n",
       "      <td>Clothing Store</td>\n",
       "      <td>120 Broadway</td>\n",
       "      <td>btw Cedar &amp; Pine</td>\n",
       "      <td>40.707970</td>\n",
       "      <td>-74.009949</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.70796968555732, 'lng': -74.00994920143108}]</td>\n",
       "      <td>900</td>\n",
       "      <td>10271</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[120 Broadway (btw Cedar &amp; Pine), New York, NY 10271, United States]</td>\n",
       "      <td>4c3b8244def90f478f0ffe2d</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Ecco</td>\n",
       "      <td>Italian Restaurant</td>\n",
       "      <td>124 Chambers St</td>\n",
       "      <td>btwn Church St &amp; W Broadway</td>\n",
       "      <td>40.715337</td>\n",
       "      <td>-74.008848</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.71533713859952, 'lng': -74.00884766217825}]</td>\n",
       "      <td>549</td>\n",
       "      <td>10007</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[124 Chambers St (btwn Church St &amp; W Broadway), New York, NY 10007, United States]</td>\n",
       "      <td>3fd66200f964a520f4e41ee3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Gloria's Restaurant</td>\n",
       "      <td>Italian Restaurant</td>\n",
       "      <td>107 W Broadway</td>\n",
       "      <td>Reade St</td>\n",
       "      <td>40.710412</td>\n",
       "      <td>-74.007998</td>\n",
       "      <td>[{'label': 'display', 'lat': 40.710412, 'lng': -74.007998}]</td>\n",
       "      <td>799</td>\n",
       "      <td>10013</td>\n",
       "      <td>US</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>United States</td>\n",
       "      <td>[107 W Broadway (Reade St), New York, NY 10013, United States]</td>\n",
       "      <td>4c5de89e6ebe2d7f7955d42e</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              name          categories          address  \\\n",
       "0  Harry's Italian Pizza Bar        Italian Restaurant  2 Gold St         \n",
       "1  Harry's Italian Pizza Bar        Pizza Place         225 Murray St     \n",
       "2  Pisillo Italian Panini           Sandwich Place      97 Nassau St      \n",
       "3  Benvenuti Italian Specialties    Gourmet Shop        235 S End Ave     \n",
       "4  Conca Cucina Italian Restaurant  Food                63 W Broadway     \n",
       "5  Pisillo Italian Cafe             Café                NaN               \n",
       "6  Caruso's Italian Restaurant      Italian Restaurant  42 Broadway       \n",
       "7  Barber Italian Shop              Clothing Store      120 Broadway      \n",
       "8  Ecco                             Italian Restaurant  124 Chambers St   \n",
       "9  Gloria's Restaurant              Italian Restaurant  107 W Broadway    \n",
       "\n",
       "                   crossStreet        lat        lng  \\\n",
       "0  at Maiden Ln.                40.707634 -74.006996   \n",
       "1  NaN                          40.715218 -74.014739   \n",
       "2  at Ann St                    40.710530 -74.007526   \n",
       "3  NaN                          40.709681 -74.016520   \n",
       "4  NaN                          40.714484 -74.009806   \n",
       "5  NaN                          40.710493 -74.007546   \n",
       "6  NaN                          40.710509 -74.008276   \n",
       "7  btw Cedar & Pine             40.707970 -74.009949   \n",
       "8  btwn Church St & W Broadway  40.715337 -74.008848   \n",
       "9  Reade St                     40.710412 -74.007998   \n",
       "\n",
       "                                                                 labeledLatLngs  \\\n",
       "0  [{'label': 'display', 'lat': 40.7076335455641, 'lng': -74.00699585960496}]     \n",
       "1  [{'label': 'display', 'lat': 40.71521779064671, 'lng': -74.01473940209351}]    \n",
       "2  [{'label': 'display', 'lat': 40.710530084460785, 'lng': -74.00752571804911}]   \n",
       "3  [{'label': 'display', 'lat': 40.70968105555279, 'lng': -74.01652043988257}]    \n",
       "4  [{'label': 'display', 'lat': 40.714484000000006, 'lng': -74.00980600000001}]   \n",
       "5  [{'label': 'display', 'lat': 40.71049280792626, 'lng': -74.00754611675153}]    \n",
       "6  [{'label': 'display', 'lat': 40.710509099999996, 'lng': -74.00827615}]         \n",
       "7  [{'label': 'display', 'lat': 40.70796968555732, 'lng': -74.00994920143108}]    \n",
       "8  [{'label': 'display', 'lat': 40.71533713859952, 'lng': -74.00884766217825}]    \n",
       "9  [{'label': 'display', 'lat': 40.710412, 'lng': -74.007998}]                    \n",
       "\n",
       "   distance postalCode  cc      city state        country  \\\n",
       "0  1076      10038      US  New York  NY    United States   \n",
       "1  58        10282      US  New York  NY    United States   \n",
       "2  822       10038      US  New York  NY    United States   \n",
       "3  595       10280      US  New York  NY    United States   \n",
       "4  469       10007      US  New York  NY    United States   \n",
       "5  823       NaN        US  New York  NY    United States   \n",
       "6  774       10004      US  New York  NY    United States   \n",
       "7  900       10271      US  New York  NY    United States   \n",
       "8  549       10007      US  New York  NY    United States   \n",
       "9  799       10013      US  New York  NY    United States   \n",
       "\n",
       "                                                                     formattedAddress  \\\n",
       "0  [2 Gold St (at Maiden Ln.), New York, NY 10038, United States]                       \n",
       "1  [225 Murray St, New York, NY 10282, United States]                                   \n",
       "2  [97 Nassau St (at Ann St), New York, NY 10038, United States]                        \n",
       "3  [235 S End Ave, New York, NY 10280, United States]                                   \n",
       "4  [63 W Broadway, New York, NY 10007, United States]                                   \n",
       "5  [New York, NY, United States]                                                        \n",
       "6  [42 Broadway, New York, NY 10004, United States]                                     \n",
       "7  [120 Broadway (btw Cedar & Pine), New York, NY 10271, United States]                 \n",
       "8  [124 Chambers St (btwn Church St & W Broadway), New York, NY 10007, United States]   \n",
       "9  [107 W Broadway (Reade St), New York, NY 10013, United States]                       \n",
       "\n",
       "                         id  \n",
       "0  4a4560a7f964a5201aa81fe3  \n",
       "1  4fa862b3e4b0ebff2f749f06  \n",
       "2  528bf16711d2b7722da6b51c  \n",
       "3  4f3feae2e4b0ac9416a72118  \n",
       "4  4f3232e219836c91c7bfde94  \n",
       "5  588a4316326c5a4b60559f17  \n",
       "6  4e4c4d28bd413c4cc66857a5  \n",
       "7  4c3b8244def90f478f0ffe2d  \n",
       "8  3fd66200f964a520f4e41ee3  \n",
       "9  4c5de89e6ebe2d7f7955d42e  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# keep only columns that include venue name, and anything that is associated with location\n",
    "filtered_columns = ['name', 'categories'] + [col for col in dataframe.columns if col.startswith('location.')] + ['id']\n",
    "dataframe_filtered = dataframe.loc[:, filtered_columns]\n",
    "\n",
    "# function that extracts the category of the venue\n",
    "def get_category_type(row):\n",
    "    try:\n",
    "        categories_list = row['categories']\n",
    "    except:\n",
    "        categories_list = row['venue.categories']\n",
    "        \n",
    "    if len(categories_list) == 0:\n",
    "        return None\n",
    "    else:\n",
    "        return categories_list[0]['name']\n",
    "\n",
    "# filter the category for each row\n",
    "dataframe_filtered['categories'] = dataframe_filtered.apply(get_category_type, axis=1)\n",
    "\n",
    "# clean column names by keeping only last term\n",
    "dataframe_filtered.columns = [column.split('.')[-1] for column in dataframe_filtered.columns]\n",
    "\n",
    "dataframe_filtered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Let's visualize the Italian restaurants that are nearby"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          Harry's Italian Pizza Bar\n",
       "1          Harry's Italian Pizza Bar\n",
       "2             Pisillo Italian Panini\n",
       "3      Benvenuti Italian Specialties\n",
       "4    Conca Cucina Italian Restaurant\n",
       "5               Pisillo Italian Cafe\n",
       "6        Caruso's Italian Restaurant\n",
       "7                Barber Italian Shop\n",
       "8                               Ecco\n",
       "9                Gloria's Restaurant\n",
       "Name: name, dtype: object"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe_filtered.name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"about:blank\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" data-html=PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgCiAgICAgICAgPHNjcmlwdD4KICAgICAgICAgICAgTF9OT19UT1VDSCA9IGZhbHNlOwogICAgICAgICAgICBMX0RJU0FCTEVfM0QgPSBmYWxzZTsKICAgICAgICA8L3NjcmlwdD4KICAgIAogICAgPHNjcmlwdCBzcmM9Imh0dHBzOi8vY2RuLmpzZGVsaXZyLm5ldC9ucG0vbGVhZmxldEAxLjUuMS9kaXN0L2xlYWZsZXQuanMiPjwvc2NyaXB0PgogICAgPHNjcmlwdCBzcmM9Imh0dHBzOi8vY29kZS5qcXVlcnkuY29tL2pxdWVyeS0xLjEyLjQubWluLmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9qcy9ib290c3RyYXAubWluLmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5qcyI+PC9zY3JpcHQ+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vY2RuLmpzZGVsaXZyLm5ldC9ucG0vbGVhZmxldEAxLjUuMS9kaXN0L2xlYWZsZXQuY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vYm9vdHN0cmFwLzMuMi4wL2Nzcy9ib290c3RyYXAubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLXRoZW1lLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9mb250LWF3ZXNvbWUvNC42LjMvY3NzL2ZvbnQtYXdlc29tZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vY2RuanMuY2xvdWRmbGFyZS5jb20vYWpheC9saWJzL0xlYWZsZXQuYXdlc29tZS1tYXJrZXJzLzIuMC4yL2xlYWZsZXQuYXdlc29tZS1tYXJrZXJzLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL3Jhd2Nkbi5naXRoYWNrLmNvbS9weXRob24tdmlzdWFsaXphdGlvbi9mb2xpdW0vbWFzdGVyL2ZvbGl1bS90ZW1wbGF0ZXMvbGVhZmxldC5hd2Vzb21lLnJvdGF0ZS5jc3MiLz4KICAgIDxzdHlsZT5odG1sLCBib2R5IHt3aWR0aDogMTAwJTtoZWlnaHQ6IDEwMCU7bWFyZ2luOiAwO3BhZGRpbmc6IDA7fTwvc3R5bGU+CiAgICA8c3R5bGU+I21hcCB7cG9zaXRpb246YWJzb2x1dGU7dG9wOjA7Ym90dG9tOjA7cmlnaHQ6MDtsZWZ0OjA7fTwvc3R5bGU+CiAgICAKICAgICAgICAgICAgPG1ldGEgbmFtZT0idmlld3BvcnQiIGNvbnRlbnQ9IndpZHRoPWRldmljZS13aWR0aCwKICAgICAgICAgICAgICAgIGluaXRpYWwtc2NhbGU9MS4wLCBtYXhpbXVtLXNjYWxlPTEuMCwgdXNlci1zY2FsYWJsZT1ubyIgLz4KICAgICAgICAgICAgPHN0eWxlPgogICAgICAgICAgICAgICAgI21hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSB7CiAgICAgICAgICAgICAgICAgICAgcG9zaXRpb246IHJlbGF0aXZlOwogICAgICAgICAgICAgICAgICAgIHdpZHRoOiAxMDAuMCU7CiAgICAgICAgICAgICAgICAgICAgaGVpZ2h0OiAxMDAuMCU7CiAgICAgICAgICAgICAgICAgICAgbGVmdDogMC4wJTsKICAgICAgICAgICAgICAgICAgICB0b3A6IDAuMCU7CiAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgIDwvc3R5bGU+CiAgICAgICAgCjwvaGVhZD4KPGJvZHk+ICAgIAogICAgCiAgICAgICAgICAgIDxkaXYgY2xhc3M9ImZvbGl1bS1tYXAiIGlkPSJtYXBfNWNmNWMzZDk1MjFjNDhhYWEyOTUxOGM4NTAzZmU2M2EiID48L2Rpdj4KICAgICAgICAKPC9ib2R5Pgo8c2NyaXB0PiAgICAKICAgIAogICAgICAgICAgICB2YXIgbWFwXzVjZjVjM2Q5NTIxYzQ4YWFhMjk1MThjODUwM2ZlNjNhID0gTC5tYXAoCiAgICAgICAgICAgICAgICAibWFwXzVjZjVjM2Q5NTIxYzQ4YWFhMjk1MThjODUwM2ZlNjNhIiwKICAgICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICAgICBjZW50ZXI6IFs0MC43MTQ5NTU1LCAtNzQuMDE1MzM2NV0sCiAgICAgICAgICAgICAgICAgICAgY3JzOiBMLkNSUy5FUFNHMzg1NywKICAgICAgICAgICAgICAgICAgICB6b29tOiAxMywKICAgICAgICAgICAgICAgICAgICB6b29tQ29udHJvbDogdHJ1ZSwKICAgICAgICAgICAgICAgICAgICBwcmVmZXJDYW52YXM6IGZhbHNlLAogICAgICAgICAgICAgICAgfQogICAgICAgICAgICApOwoKICAgICAgICAgICAgCgogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciB0aWxlX2xheWVyX2FlMDdlOTBhYWY0YjQ2NjY4YmRhNDllNWY0MDk5Yzk2ID0gTC50aWxlTGF5ZXIoCiAgICAgICAgICAgICAgICAiaHR0cHM6Ly97c30udGlsZS5vcGVuc3RyZWV0bWFwLm9yZy97en0ve3h9L3t5fS5wbmciLAogICAgICAgICAgICAgICAgeyJhdHRyaWJ1dGlvbiI6ICJEYXRhIGJ5IFx1MDAyNmNvcHk7IFx1MDAzY2EgaHJlZj1cImh0dHA6Ly9vcGVuc3RyZWV0bWFwLm9yZ1wiXHUwMDNlT3BlblN0cmVldE1hcFx1MDAzYy9hXHUwMDNlLCB1bmRlciBcdTAwM2NhIGhyZWY9XCJodHRwOi8vd3d3Lm9wZW5zdHJlZXRtYXAub3JnL2NvcHlyaWdodFwiXHUwMDNlT0RiTFx1MDAzYy9hXHUwMDNlLiIsICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwgIm1heE5hdGl2ZVpvb20iOiAxOCwgIm1heFpvb20iOiAxOCwgIm1pblpvb20iOiAwLCAibm9XcmFwIjogZmFsc2UsICJvcGFjaXR5IjogMSwgInN1YmRvbWFpbnMiOiAiYWJjIiwgInRtcyI6IGZhbHNlfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfNzYxZGZkM2Y2YWI3NDhiNGE2MDAyYmYxODkyZjEyZDIgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTQ5NTU1LCAtNzQuMDE1MzM2NV0sCiAgICAgICAgICAgICAgICB7ImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLCAiY29sb3IiOiAicmVkIiwgImRhc2hBcnJheSI6IG51bGwsICJkYXNoT2Zmc2V0IjogbnVsbCwgImZpbGwiOiB0cnVlLCAiZmlsbENvbG9yIjogInJlZCIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogMTAsICJzdHJva2UiOiB0cnVlLCAid2VpZ2h0IjogM30KICAgICAgICAgICAgKS5hZGRUbyhtYXBfNWNmNWMzZDk1MjFjNDhhYWEyOTUxOGM4NTAzZmU2M2EpOwogICAgICAgIAogICAgCiAgICAgICAgdmFyIHBvcHVwXzE3YzcwZWNmMmJjYjQ3NmZiNTM1MjY2OTAzOGY4N2VjID0gTC5wb3B1cCh7Im1heFdpZHRoIjogIjEwMCUifSk7CgogICAgICAgIAogICAgICAgICAgICB2YXIgaHRtbF81YTIzYjg2NWE4YzM0M2VhYmU0NTE2ZmMzNTllNGFkNyA9ICQoYDxkaXYgaWQ9Imh0bWxfNWEyM2I4NjVhOGMzNDNlYWJlNDUxNmZjMzU5ZTRhZDciIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkNvbnJhZCBIb3RlbDwvZGl2PmApWzBdOwogICAgICAgICAgICBwb3B1cF8xN2M3MGVjZjJiY2I0NzZmYjUzNTI2NjkwMzhmODdlYy5zZXRDb250ZW50KGh0bWxfNWEyM2I4NjVhOGMzNDNlYWJlNDUxNmZjMzU5ZTRhZDcpOwogICAgICAgIAoKICAgICAgICBjaXJjbGVfbWFya2VyXzc2MWRmZDNmNmFiNzQ4YjRhNjAwMmJmMTg5MmYxMmQyLmJpbmRQb3B1cChwb3B1cF8xN2M3MGVjZjJiY2I0NzZmYjUzNTI2NjkwMzhmODdlYykKICAgICAgICA7CgogICAgICAgIAogICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfZjAzZjIwOGJlNjU3NGFjOWEzNGM0MzFkYTc5NTkwMjUgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MDc2MzM1NDU1NjQxLCAtNzQuMDA2OTk1ODU5NjA0OTZdLAogICAgICAgICAgICAgICAgeyJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwgImNvbG9yIjogImJsdWUiLCAiZGFzaEFycmF5IjogbnVsbCwgImRhc2hPZmZzZXQiOiBudWxsLCAiZmlsbCI6IHRydWUsICJmaWxsQ29sb3IiOiAiYmx1ZSIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogNSwgInN0cm9rZSI6IHRydWUsICJ3ZWlnaHQiOiAzfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICB2YXIgcG9wdXBfMzE5MWIwYzY0OGYwNDM4MTg2MjNkYTIyZDRlNmU0YWMgPSBMLnBvcHVwKHsibWF4V2lkdGgiOiAiMTAwJSJ9KTsKCiAgICAgICAgCiAgICAgICAgICAgIHZhciBodG1sXzdiODY4YTVhMGZkZDRlODVhMjRlOTAyZDA1YzExNzBmID0gJChgPGRpdiBpZD0iaHRtbF83Yjg2OGE1YTBmZGQ0ZTg1YTI0ZTkwMmQwNWMxMTcwZiIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+SXRhbGlhbiBSZXN0YXVyYW50PC9kaXY+YClbMF07CiAgICAgICAgICAgIHBvcHVwXzMxOTFiMGM2NDhmMDQzODE4NjIzZGEyMmQ0ZTZlNGFjLnNldENvbnRlbnQoaHRtbF83Yjg2OGE1YTBmZGQ0ZTg1YTI0ZTkwMmQwNWMxMTcwZik7CiAgICAgICAgCgogICAgICAgIGNpcmNsZV9tYXJrZXJfZjAzZjIwOGJlNjU3NGFjOWEzNGM0MzFkYTc5NTkwMjUuYmluZFBvcHVwKHBvcHVwXzMxOTFiMGM2NDhmMDQzODE4NjIzZGEyMmQ0ZTZlNGFjKQogICAgICAgIDsKCiAgICAgICAgCiAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl9hMzhiNDc5YWY1ZmY0NTcwYjU4ZTg2OGFjMjIyYjgzNyA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTIxNzc5MDY0NjcxLCAtNzQuMDE0NzM5NDAyMDkzNTFdLAogICAgICAgICAgICAgICAgeyJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwgImNvbG9yIjogImJsdWUiLCAiZGFzaEFycmF5IjogbnVsbCwgImRhc2hPZmZzZXQiOiBudWxsLCAiZmlsbCI6IHRydWUsICJmaWxsQ29sb3IiOiAiYmx1ZSIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogNSwgInN0cm9rZSI6IHRydWUsICJ3ZWlnaHQiOiAzfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICB2YXIgcG9wdXBfODRmMmY0MTExMDM5NDE4OTgzOWExNTQ4YjI5ZGUwYjIgPSBMLnBvcHVwKHsibWF4V2lkdGgiOiAiMTAwJSJ9KTsKCiAgICAgICAgCiAgICAgICAgICAgIHZhciBodG1sX2MwNzkzYThmYzdkOTRlNGFiOGNkM2NjOTY1YTAwMzIzID0gJChgPGRpdiBpZD0iaHRtbF9jMDc5M2E4ZmM3ZDk0ZTRhYjhjZDNjYzk2NWEwMDMyMyIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+UGl6emEgUGxhY2U8L2Rpdj5gKVswXTsKICAgICAgICAgICAgcG9wdXBfODRmMmY0MTExMDM5NDE4OTgzOWExNTQ4YjI5ZGUwYjIuc2V0Q29udGVudChodG1sX2MwNzkzYThmYzdkOTRlNGFiOGNkM2NjOTY1YTAwMzIzKTsKICAgICAgICAKCiAgICAgICAgY2lyY2xlX21hcmtlcl9hMzhiNDc5YWY1ZmY0NTcwYjU4ZTg2OGFjMjIyYjgzNy5iaW5kUG9wdXAocG9wdXBfODRmMmY0MTExMDM5NDE4OTgzOWExNTQ4YjI5ZGUwYjIpCiAgICAgICAgOwoKICAgICAgICAKICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyX2YyMTdiNzA0Mzg0ZjQzNmFiOWEwOWM2MGU2YmQzM2JhID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzEwNTMwMDg0NDYwNzg1LCAtNzQuMDA3NTI1NzE4MDQ5MTFdLAogICAgICAgICAgICAgICAgeyJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwgImNvbG9yIjogImJsdWUiLCAiZGFzaEFycmF5IjogbnVsbCwgImRhc2hPZmZzZXQiOiBudWxsLCAiZmlsbCI6IHRydWUsICJmaWxsQ29sb3IiOiAiYmx1ZSIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogNSwgInN0cm9rZSI6IHRydWUsICJ3ZWlnaHQiOiAzfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICB2YXIgcG9wdXBfNWNhMWRiOWI5YzkzNDkyZDhlNjE2YTJlMGRlZjk0MGUgPSBMLnBvcHVwKHsibWF4V2lkdGgiOiAiMTAwJSJ9KTsKCiAgICAgICAgCiAgICAgICAgICAgIHZhciBodG1sXzMxYTc1YzRjYjQ5MTRjYmI5N2NkMjdhZWNmNDBjYTVmID0gJChgPGRpdiBpZD0iaHRtbF8zMWE3NWM0Y2I0OTE0Y2JiOTdjZDI3YWVjZjQwY2E1ZiIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+U2FuZHdpY2ggUGxhY2U8L2Rpdj5gKVswXTsKICAgICAgICAgICAgcG9wdXBfNWNhMWRiOWI5YzkzNDkyZDhlNjE2YTJlMGRlZjk0MGUuc2V0Q29udGVudChodG1sXzMxYTc1YzRjYjQ5MTRjYmI5N2NkMjdhZWNmNDBjYTVmKTsKICAgICAgICAKCiAgICAgICAgY2lyY2xlX21hcmtlcl9mMjE3YjcwNDM4NGY0MzZhYjlhMDljNjBlNmJkMzNiYS5iaW5kUG9wdXAocG9wdXBfNWNhMWRiOWI5YzkzNDkyZDhlNjE2YTJlMGRlZjk0MGUpCiAgICAgICAgOwoKICAgICAgICAKICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyX2NjMDY1MzJkYTNlMDQzNmFiMDQ5Y2JmZjZkYzI3MjQzID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzA5NjgxMDU1NTUyNzksIC03NC4wMTY1MjA0Mzk4ODI1N10sCiAgICAgICAgICAgICAgICB7ImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLCAiY29sb3IiOiAiYmx1ZSIsICJkYXNoQXJyYXkiOiBudWxsLCAiZGFzaE9mZnNldCI6IG51bGwsICJmaWxsIjogdHJ1ZSwgImZpbGxDb2xvciI6ICJibHVlIiwgImZpbGxPcGFjaXR5IjogMC42LCAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsICJsaW5lQ2FwIjogInJvdW5kIiwgImxpbmVKb2luIjogInJvdW5kIiwgIm9wYWNpdHkiOiAxLjAsICJyYWRpdXMiOiA1LCAic3Ryb2tlIjogdHJ1ZSwgIndlaWdodCI6IDN9CiAgICAgICAgICAgICkuYWRkVG8obWFwXzVjZjVjM2Q5NTIxYzQ4YWFhMjk1MThjODUwM2ZlNjNhKTsKICAgICAgICAKICAgIAogICAgICAgIHZhciBwb3B1cF8yM2U0MWRkNWYyMTI0NjNmOThjZTcxYzE4MzYyMDVkNCA9IEwucG9wdXAoeyJtYXhXaWR0aCI6ICIxMDAlIn0pOwoKICAgICAgICAKICAgICAgICAgICAgdmFyIGh0bWxfZDkzODhkNWE5ZjFlNDQ0ZWJmNDAxNWM2YmFkY2UzZTggPSAkKGA8ZGl2IGlkPSJodG1sX2Q5Mzg4ZDVhOWYxZTQ0NGViZjQwMTVjNmJhZGNlM2U4IiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5Hb3VybWV0IFNob3A8L2Rpdj5gKVswXTsKICAgICAgICAgICAgcG9wdXBfMjNlNDFkZDVmMjEyNDYzZjk4Y2U3MWMxODM2MjA1ZDQuc2V0Q29udGVudChodG1sX2Q5Mzg4ZDVhOWYxZTQ0NGViZjQwMTVjNmJhZGNlM2U4KTsKICAgICAgICAKCiAgICAgICAgY2lyY2xlX21hcmtlcl9jYzA2NTMyZGEzZTA0MzZhYjA0OWNiZmY2ZGMyNzI0My5iaW5kUG9wdXAocG9wdXBfMjNlNDFkZDVmMjEyNDYzZjk4Y2U3MWMxODM2MjA1ZDQpCiAgICAgICAgOwoKICAgICAgICAKICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzE3ZjZkZjgwYjM5NjQ2YTk4N2ZhNmMzMmJhYjA3MWMzID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE0NDg0MDAwMDAwMDA2LCAtNzQuMDA5ODA2MDAwMDAwMDFdLAogICAgICAgICAgICAgICAgeyJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwgImNvbG9yIjogImJsdWUiLCAiZGFzaEFycmF5IjogbnVsbCwgImRhc2hPZmZzZXQiOiBudWxsLCAiZmlsbCI6IHRydWUsICJmaWxsQ29sb3IiOiAiYmx1ZSIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogNSwgInN0cm9rZSI6IHRydWUsICJ3ZWlnaHQiOiAzfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICB2YXIgcG9wdXBfYmM2YTY1YjE2ZTE0NDFkYTk2MzMwZTY3MzkwM2IwNDggPSBMLnBvcHVwKHsibWF4V2lkdGgiOiAiMTAwJSJ9KTsKCiAgICAgICAgCiAgICAgICAgICAgIHZhciBodG1sX2Q2YThlZDk5M2ExYTQyYWU4ZTY2NjUyYTE3ODQwYmNhID0gJChgPGRpdiBpZD0iaHRtbF9kNmE4ZWQ5OTNhMWE0MmFlOGU2NjY1MmExNzg0MGJjYSIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+Rm9vZDwvZGl2PmApWzBdOwogICAgICAgICAgICBwb3B1cF9iYzZhNjViMTZlMTQ0MWRhOTYzMzBlNjczOTAzYjA0OC5zZXRDb250ZW50KGh0bWxfZDZhOGVkOTkzYTFhNDJhZThlNjY2NTJhMTc4NDBiY2EpOwogICAgICAgIAoKICAgICAgICBjaXJjbGVfbWFya2VyXzE3ZjZkZjgwYjM5NjQ2YTk4N2ZhNmMzMmJhYjA3MWMzLmJpbmRQb3B1cChwb3B1cF9iYzZhNjViMTZlMTQ0MWRhOTYzMzBlNjczOTAzYjA0OCkKICAgICAgICA7CgogICAgICAgIAogICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfMDQ4NDQyNWQzNGIyNGEzZWIyYjAzM2EyNDRlYjllNDUgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTA0OTI4MDc5MjYyNiwgLTc0LjAwNzU0NjExNjc1MTUzXSwKICAgICAgICAgICAgICAgIHsiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsICJjb2xvciI6ICJibHVlIiwgImRhc2hBcnJheSI6IG51bGwsICJkYXNoT2Zmc2V0IjogbnVsbCwgImZpbGwiOiB0cnVlLCAiZmlsbENvbG9yIjogImJsdWUiLCAiZmlsbE9wYWNpdHkiOiAwLjYsICJmaWxsUnVsZSI6ICJldmVub2RkIiwgImxpbmVDYXAiOiAicm91bmQiLCAibGluZUpvaW4iOiAicm91bmQiLCAib3BhY2l0eSI6IDEuMCwgInJhZGl1cyI6IDUsICJzdHJva2UiOiB0cnVlLCAid2VpZ2h0IjogM30KICAgICAgICAgICAgKS5hZGRUbyhtYXBfNWNmNWMzZDk1MjFjNDhhYWEyOTUxOGM4NTAzZmU2M2EpOwogICAgICAgIAogICAgCiAgICAgICAgdmFyIHBvcHVwXzM4MDBkMTMyZjRlMDQwZDI5NWJkYzlhNzFjZThhODAwID0gTC5wb3B1cCh7Im1heFdpZHRoIjogIjEwMCUifSk7CgogICAgICAgIAogICAgICAgICAgICB2YXIgaHRtbF9mZWRlNWMyOWM1NmQ0N2Q3ODFjZTFmMjNkMzZjYzA2ZSA9ICQoYDxkaXYgaWQ9Imh0bWxfZmVkZTVjMjljNTZkNDdkNzgxY2UxZjIzZDM2Y2MwNmUiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkNhZsOpPC9kaXY+YClbMF07CiAgICAgICAgICAgIHBvcHVwXzM4MDBkMTMyZjRlMDQwZDI5NWJkYzlhNzFjZThhODAwLnNldENvbnRlbnQoaHRtbF9mZWRlNWMyOWM1NmQ0N2Q3ODFjZTFmMjNkMzZjYzA2ZSk7CiAgICAgICAgCgogICAgICAgIGNpcmNsZV9tYXJrZXJfMDQ4NDQyNWQzNGIyNGEzZWIyYjAzM2EyNDRlYjllNDUuYmluZFBvcHVwKHBvcHVwXzM4MDBkMTMyZjRlMDQwZDI5NWJkYzlhNzFjZThhODAwKQogICAgICAgIDsKCiAgICAgICAgCiAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8yODg0MjFiYjExZDY0NTQ3YjUwZTU1NGY5MDhhM2RhOCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxMDUwOTA5OTk5OTk5NiwgLTc0LjAwODI3NjE1XSwKICAgICAgICAgICAgICAgIHsiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsICJjb2xvciI6ICJibHVlIiwgImRhc2hBcnJheSI6IG51bGwsICJkYXNoT2Zmc2V0IjogbnVsbCwgImZpbGwiOiB0cnVlLCAiZmlsbENvbG9yIjogImJsdWUiLCAiZmlsbE9wYWNpdHkiOiAwLjYsICJmaWxsUnVsZSI6ICJldmVub2RkIiwgImxpbmVDYXAiOiAicm91bmQiLCAibGluZUpvaW4iOiAicm91bmQiLCAib3BhY2l0eSI6IDEuMCwgInJhZGl1cyI6IDUsICJzdHJva2UiOiB0cnVlLCAid2VpZ2h0IjogM30KICAgICAgICAgICAgKS5hZGRUbyhtYXBfNWNmNWMzZDk1MjFjNDhhYWEyOTUxOGM4NTAzZmU2M2EpOwogICAgICAgIAogICAgCiAgICAgICAgdmFyIHBvcHVwX2Q2YWMxOGYzNzM0NjQxYjM5NGI0M2I4ZmJlOWU2YTZiID0gTC5wb3B1cCh7Im1heFdpZHRoIjogIjEwMCUifSk7CgogICAgICAgIAogICAgICAgICAgICB2YXIgaHRtbF9lMDk0NTNlMDMzZjk0OTNhYjg3YWI5YjNkYjEwNTQ0MSA9ICQoYDxkaXYgaWQ9Imh0bWxfZTA5NDUzZTAzM2Y5NDkzYWI4N2FiOWIzZGIxMDU0NDEiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkl0YWxpYW4gUmVzdGF1cmFudDwvZGl2PmApWzBdOwogICAgICAgICAgICBwb3B1cF9kNmFjMThmMzczNDY0MWIzOTRiNDNiOGZiZTllNmE2Yi5zZXRDb250ZW50KGh0bWxfZTA5NDUzZTAzM2Y5NDkzYWI4N2FiOWIzZGIxMDU0NDEpOwogICAgICAgIAoKICAgICAgICBjaXJjbGVfbWFya2VyXzI4ODQyMWJiMTFkNjQ1NDdiNTBlNTU0ZjkwOGEzZGE4LmJpbmRQb3B1cChwb3B1cF9kNmFjMThmMzczNDY0MWIzOTRiNDNiOGZiZTllNmE2YikKICAgICAgICA7CgogICAgICAgIAogICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfOTYxNTk1MDIwYWJkNGY1MDkxYjM2MzA5ODAwYjNiYTMgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MDc5Njk2ODU1NTczMiwgLTc0LjAwOTk0OTIwMTQzMTA4XSwKICAgICAgICAgICAgICAgIHsiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsICJjb2xvciI6ICJibHVlIiwgImRhc2hBcnJheSI6IG51bGwsICJkYXNoT2Zmc2V0IjogbnVsbCwgImZpbGwiOiB0cnVlLCAiZmlsbENvbG9yIjogImJsdWUiLCAiZmlsbE9wYWNpdHkiOiAwLjYsICJmaWxsUnVsZSI6ICJldmVub2RkIiwgImxpbmVDYXAiOiAicm91bmQiLCAibGluZUpvaW4iOiAicm91bmQiLCAib3BhY2l0eSI6IDEuMCwgInJhZGl1cyI6IDUsICJzdHJva2UiOiB0cnVlLCAid2VpZ2h0IjogM30KICAgICAgICAgICAgKS5hZGRUbyhtYXBfNWNmNWMzZDk1MjFjNDhhYWEyOTUxOGM4NTAzZmU2M2EpOwogICAgICAgIAogICAgCiAgICAgICAgdmFyIHBvcHVwX2U4MDAzNDAwYzVkMDRjNzY5NzdmYzkxZmViYWViZWE1ID0gTC5wb3B1cCh7Im1heFdpZHRoIjogIjEwMCUifSk7CgogICAgICAgIAogICAgICAgICAgICB2YXIgaHRtbF9iNzZlNTc5ZTZiMzM0YTdmYTUzOTU2YmFkMTRhYjA3MiA9ICQoYDxkaXYgaWQ9Imh0bWxfYjc2ZTU3OWU2YjMzNGE3ZmE1Mzk1NmJhZDE0YWIwNzIiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkNsb3RoaW5nIFN0b3JlPC9kaXY+YClbMF07CiAgICAgICAgICAgIHBvcHVwX2U4MDAzNDAwYzVkMDRjNzY5NzdmYzkxZmViYWViZWE1LnNldENvbnRlbnQoaHRtbF9iNzZlNTc5ZTZiMzM0YTdmYTUzOTU2YmFkMTRhYjA3Mik7CiAgICAgICAgCgogICAgICAgIGNpcmNsZV9tYXJrZXJfOTYxNTk1MDIwYWJkNGY1MDkxYjM2MzA5ODAwYjNiYTMuYmluZFBvcHVwKHBvcHVwX2U4MDAzNDAwYzVkMDRjNzY5NzdmYzkxZmViYWViZWE1KQogICAgICAgIDsKCiAgICAgICAgCiAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8wYmRjMDg0ZmUxY2I0NmQwYWMyOWQ0NmM1ZmNlYWNjZiA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTMzNzEzODU5OTUyLCAtNzQuMDA4ODQ3NjYyMTc4MjVdLAogICAgICAgICAgICAgICAgeyJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwgImNvbG9yIjogImJsdWUiLCAiZGFzaEFycmF5IjogbnVsbCwgImRhc2hPZmZzZXQiOiBudWxsLCAiZmlsbCI6IHRydWUsICJmaWxsQ29sb3IiOiAiYmx1ZSIsICJmaWxsT3BhY2l0eSI6IDAuNiwgImZpbGxSdWxlIjogImV2ZW5vZGQiLCAibGluZUNhcCI6ICJyb3VuZCIsICJsaW5lSm9pbiI6ICJyb3VuZCIsICJvcGFjaXR5IjogMS4wLCAicmFkaXVzIjogNSwgInN0cm9rZSI6IHRydWUsICJ3ZWlnaHQiOiAzfQogICAgICAgICAgICApLmFkZFRvKG1hcF81Y2Y1YzNkOTUyMWM0OGFhYTI5NTE4Yzg1MDNmZTYzYSk7CiAgICAgICAgCiAgICAKICAgICAgICB2YXIgcG9wdXBfNjUzNzhkYzQzYTdmNDQwZmI5Nzc1MTVkMGE5MzMzMGUgPSBMLnBvcHVwKHsibWF4V2lkdGgiOiAiMTAwJSJ9KTsKCiAgICAgICAgCiAgICAgICAgICAgIHZhciBodG1sXzg0M2NhMGE1ZjczMzQ4OTBhN2ZhYjUwNjU0ODI5ZGY4ID0gJChgPGRpdiBpZD0iaHRtbF84NDNjYTBhNWY3MzM0ODkwYTdmYWI1MDY1NDgyOWRmOCIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+SXRhbGlhbiBSZXN0YXVyYW50PC9kaXY+YClbMF07CiAgICAgICAgICAgIHBvcHVwXzY1Mzc4ZGM0M2E3ZjQ0MGZiOTc3NTE1ZDBhOTMzMzBlLnNldENvbnRlbnQoaHRtbF84NDNjYTBhNWY3MzM0ODkwYTdmYWI1MDY1NDgyOWRmOCk7CiAgICAgICAgCgogICAgICAgIGNpcmNsZV9tYXJrZXJfMGJkYzA4NGZlMWNiNDZkMGFjMjlkNDZjNWZjZWFjY2YuYmluZFBvcHVwKHBvcHVwXzY1Mzc4ZGM0M2E3ZjQ0MGZiOTc3NTE1ZDBhOTMzMzBlKQogICAgICAgIDsKCiAgICAgICAgCiAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl9kYjk2MTU3MTk1NGI0ZDk0OTE2YjVmNGQ0MWZkMTc3OCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxMDQxMiwgLTc0LjAwNzk5OF0sCiAgICAgICAgICAgICAgICB7ImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLCAiY29sb3IiOiAiYmx1ZSIsICJkYXNoQXJyYXkiOiBudWxsLCAiZGFzaE9mZnNldCI6IG51bGwsICJmaWxsIjogdHJ1ZSwgImZpbGxDb2xvciI6ICJibHVlIiwgImZpbGxPcGFjaXR5IjogMC42LCAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsICJsaW5lQ2FwIjogInJvdW5kIiwgImxpbmVKb2luIjogInJvdW5kIiwgIm9wYWNpdHkiOiAxLjAsICJyYWRpdXMiOiA1LCAic3Ryb2tlIjogdHJ1ZSwgIndlaWdodCI6IDN9CiAgICAgICAgICAgICkuYWRkVG8obWFwXzVjZjVjM2Q5NTIxYzQ4YWFhMjk1MThjODUwM2ZlNjNhKTsKICAgICAgICAKICAgIAogICAgICAgIHZhciBwb3B1cF9lZDgxOGUzODhkYTE0MWZmOWIwNTczNDJmMTZlMjk2NCA9IEwucG9wdXAoeyJtYXhXaWR0aCI6ICIxMDAlIn0pOwoKICAgICAgICAKICAgICAgICAgICAgdmFyIGh0bWxfZThjYTI2YmYxZTg0NDY0Mjg0NTQ1MjE2MzBiZTlhMjMgPSAkKGA8ZGl2IGlkPSJodG1sX2U4Y2EyNmJmMWU4NDQ2NDI4NDU0NTIxNjMwYmU5YTIzIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5JdGFsaWFuIFJlc3RhdXJhbnQ8L2Rpdj5gKVswXTsKICAgICAgICAgICAgcG9wdXBfZWQ4MThlMzg4ZGExNDFmZjliMDU3MzQyZjE2ZTI5NjQuc2V0Q29udGVudChodG1sX2U4Y2EyNmJmMWU4NDQ2NDI4NDU0NTIxNjMwYmU5YTIzKTsKICAgICAgICAKCiAgICAgICAgY2lyY2xlX21hcmtlcl9kYjk2MTU3MTk1NGI0ZDk0OTE2YjVmNGQ0MWZkMTc3OC5iaW5kUG9wdXAocG9wdXBfZWQ4MThlMzg4ZGExNDFmZjliMDU3MzQyZjE2ZTI5NjQpCiAgICAgICAgOwoKICAgICAgICAKICAgIAo8L3NjcmlwdD4= onload=\"this.contentDocument.open();this.contentDocument.write(atob(this.getAttribute('data-html')));this.contentDocument.close();\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
      ],
      "text/plain": [
       "<folium.folium.Map at 0xf1a4880>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "venues_map = folium.Map(location=[latitude, longitude], zoom_start=13) # generate map centred around the Conrad Hotel\n",
    "\n",
    "# add a red circle marker to represent the Conrad Hotel\n",
    "folium.vector_layers.CircleMarker(\n",
    "    [latitude, longitude],\n",
    "    radius=10,\n",
    "    color='red',\n",
    "    popup='Conrad Hotel',\n",
    "    fill = True,\n",
    "    fill_color = 'red',\n",
    "    fill_opacity = 0.6\n",
    ").add_to(venues_map)\n",
    "\n",
    "# add the Italian restaurants as blue circle markers\n",
    "for lat, lng, label in zip(dataframe_filtered.lat, dataframe_filtered.lng, dataframe_filtered.categories):\n",
    "    folium.vector_layers.CircleMarker(\n",
    "        [lat, lng],\n",
    "        radius=5,\n",
    "        color='blue',\n",
    "        popup=label,\n",
    "        fill = True,\n",
    "        fill_color='blue',\n",
    "        fill_opacity=0.6\n",
    "    ).add_to(venues_map)\n",
    "\n",
    "# display map\n",
    "venues_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item2\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## 2. Explore a Given Venue\n",
    "> `https://api.foursquare.com/v2/venues/`**VENUE_ID**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### A. Let's explore the closest Italian restaurant -- _Harry's Italian Pizza Bar_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://api.foursquare.com/v2/venues/4fa862b3e4b0ebff2f749f06?client_id=H0IZ23LNW3Y4DDUGZEPW4SBIVS2QCQE0SKLAMP15UZMQGPPU&client_secret=IUSOYUQKPVUKYOYYP4OV1WB1BPOJ4IGI0R0JBS4UPDPEIYFS&v=20201103'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "venue_id = '4fa862b3e4b0ebff2f749f06' # ID of Harry's Italian Pizza Bar\n",
    "url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n",
    "url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Send GET request for result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['id', 'name', 'contact', 'location', 'canonicalUrl', 'categories', 'verified', 'stats', 'url', 'price', 'hasMenu', 'likes', 'dislike', 'ok', 'rating', 'ratingColor', 'ratingSignals', 'delivery', 'menu', 'allowMenuUrlEdit', 'beenHere', 'specials', 'photos', 'reasons', 'hereNow', 'createdAt', 'tips', 'shortUrl', 'timeZone', 'listed', 'hours', 'popular', 'pageUpdates', 'inbox', 'attributes', 'bestPhoto', 'colors'])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'id': '4fa862b3e4b0ebff2f749f06',\n",
       " 'name': \"Harry's Italian Pizza Bar\",\n",
       " 'contact': {'phone': '2126081007', 'formattedPhone': '(212) 608-1007'},\n",
       " 'location': {'address': '225 Murray St',\n",
       "  'lat': 40.71521779064671,\n",
       "  'lng': -74.01473940209351,\n",
       "  'labeledLatLngs': [{'label': 'display',\n",
       "    'lat': 40.71521779064671,\n",
       "    'lng': -74.01473940209351}],\n",
       "  'postalCode': '10282',\n",
       "  'cc': 'US',\n",
       "  'city': 'New York',\n",
       "  'state': 'NY',\n",
       "  'country': 'United States',\n",
       "  'formattedAddress': ['225 Murray St',\n",
       "   'New York, NY 10282',\n",
       "   'United States']},\n",
       " 'canonicalUrl': 'https://foursquare.com/v/harrys-italian-pizza-bar/4fa862b3e4b0ebff2f749f06',\n",
       " 'categories': [{'id': '4bf58dd8d48988d1ca941735',\n",
       "   'name': 'Pizza Place',\n",
       "   'pluralName': 'Pizza Places',\n",
       "   'shortName': 'Pizza',\n",
       "   'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/pizza_',\n",
       "    'suffix': '.png'},\n",
       "   'primary': True},\n",
       "  {'id': '4bf58dd8d48988d110941735',\n",
       "   'name': 'Italian Restaurant',\n",
       "   'pluralName': 'Italian Restaurants',\n",
       "   'shortName': 'Italian',\n",
       "   'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n",
       "    'suffix': '.png'}}],\n",
       " 'verified': False,\n",
       " 'stats': {'tipCount': 57},\n",
       " 'url': 'http://harrysitalian.com',\n",
       " 'price': {'tier': 2, 'message': 'Moderate', 'currency': '$'},\n",
       " 'hasMenu': True,\n",
       " 'likes': {'count': 120,\n",
       "  'groups': [{'type': 'others', 'count': 120, 'items': []}],\n",
       "  'summary': '120 Likes'},\n",
       " 'dislike': False,\n",
       " 'ok': False,\n",
       " 'rating': 7.0,\n",
       " 'ratingColor': 'C5DE35',\n",
       " 'ratingSignals': 212,\n",
       " 'delivery': {'id': '294544',\n",
       "  'url': 'https://www.seamless.com/menu/harrys-italian-pizza-bar-225-murray-st-new-york/294544?affiliate=1131&utm_source=foursquare-affiliate-network&utm_medium=affiliate&utm_campaign=1131&utm_content=294544',\n",
       "  'provider': {'name': 'seamless',\n",
       "   'icon': {'prefix': 'https://fastly.4sqi.net/img/general/cap/',\n",
       "    'sizes': [40, 50],\n",
       "    'name': '/delivery_provider_seamless_20180129.png'}}},\n",
       " 'menu': {'type': 'Menu',\n",
       "  'label': 'Menu',\n",
       "  'anchor': 'View Menu',\n",
       "  'url': 'https://foursquare.com/v/harrys-italian-pizza-bar/4fa862b3e4b0ebff2f749f06/menu',\n",
       "  'mobileUrl': 'https://foursquare.com/v/4fa862b3e4b0ebff2f749f06/device_menu'},\n",
       " 'allowMenuUrlEdit': True,\n",
       " 'beenHere': {'count': 0,\n",
       "  'unconfirmedCount': 0,\n",
       "  'marked': False,\n",
       "  'lastCheckinExpiredAt': 0},\n",
       " 'specials': {'count': 0, 'items': []},\n",
       " 'photos': {'count': 147,\n",
       "  'groups': [{'type': 'venue',\n",
       "    'name': 'Venue photos',\n",
       "    'count': 147,\n",
       "    'items': [{'id': '4fad980de4b091b4626c3633',\n",
       "      'createdAt': 1336776717,\n",
       "      'source': {'name': 'Foursquare for Android',\n",
       "       'url': 'https://foursquare.com/download/#/android'},\n",
       "      'prefix': 'https://fastly.4sqi.net/img/general/',\n",
       "      'suffix': '/ya1iQFI7pLjuIJp1PGDKlrZS3OJdHCF7tpILMmjv_2w.jpg',\n",
       "      'width': 480,\n",
       "      'height': 640,\n",
       "      'user': {'id': '13676709',\n",
       "       'firstName': 'Leony',\n",
       "       'lastName': 'N',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/T0ANFNGNMCHUDEUE.jpg'}},\n",
       "      'visibility': 'public'}]}]},\n",
       " 'reasons': {'count': 1,\n",
       "  'items': [{'summary': 'Lots of people like this place',\n",
       "    'type': 'general',\n",
       "    'reasonName': 'rawLikesReason'}]},\n",
       " 'hereNow': {'count': 0, 'summary': 'Nobody here', 'groups': []},\n",
       " 'createdAt': 1336435379,\n",
       " 'tips': {'count': 57,\n",
       "  'groups': [{'type': 'others',\n",
       "    'name': 'All tips',\n",
       "    'count': 57,\n",
       "    'items': [{'id': '53d27909498e0523841340b6',\n",
       "      'createdAt': 1406302473,\n",
       "      'text': \"Harry's Italian Pizza bar is known for it's amazing pizza, but did you know that the brunches here are amazing too? Try the Nutella French toast and we know you'll be sold.\",\n",
       "      'type': 'user',\n",
       "      'canonicalUrl': 'https://foursquare.com/item/53d27909498e0523841340b6',\n",
       "      'lang': 'en',\n",
       "      'likes': {'count': 4,\n",
       "       'groups': [{'type': 'others',\n",
       "         'count': 4,\n",
       "         'items': [{'id': '369426',\n",
       "           'firstName': 'P.',\n",
       "           'lastName': 'M',\n",
       "           'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "            'suffix': '/JPQYUWJKUT0H2OO4.jpg'}},\n",
       "          {'id': '87587879',\n",
       "           'firstName': 'Diane',\n",
       "           'lastName': 'D',\n",
       "           'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "            'suffix': '/87587879-ESLRSZLQ2CBE2P4W.jpg'}},\n",
       "          {'id': '87591341',\n",
       "           'firstName': 'Tim',\n",
       "           'lastName': 'S',\n",
       "           'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "            'suffix': '/-Z4YK4VKE0JSVXIY1.jpg'}},\n",
       "          {'id': '87473404',\n",
       "           'firstName': 'TenantKing.com',\n",
       "           'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "            'suffix': '/87473404-HI5DTBTK0HX401CA.png'},\n",
       "           'type': 'page'}]}],\n",
       "       'summary': '4 likes'},\n",
       "      'logView': True,\n",
       "      'agreeCount': 4,\n",
       "      'disagreeCount': 0,\n",
       "      'todo': {'count': 0},\n",
       "      'user': {'id': '87473404',\n",
       "       'firstName': 'TenantKing.com',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/87473404-HI5DTBTK0HX401CA.png'},\n",
       "       'type': 'page'}}]}]},\n",
       " 'shortUrl': 'http://4sq.com/JNblHV',\n",
       " 'timeZone': 'America/New_York',\n",
       " 'listed': {'count': 54,\n",
       "  'groups': [{'type': 'others',\n",
       "    'name': 'Lists from other people',\n",
       "    'count': 54,\n",
       "    'items': [{'id': '4fa32fd0e4b04193744746b1',\n",
       "      'name': 'Manhattan Haunts',\n",
       "      'description': '',\n",
       "      'type': 'others',\n",
       "      'user': {'id': '24592223',\n",
       "       'firstName': 'Becca',\n",
       "       'lastName': 'M',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/24592223-RAW2UYM0GIB1U40K.jpg'}},\n",
       "      'editable': False,\n",
       "      'public': True,\n",
       "      'collaborative': False,\n",
       "      'url': '/becca_mcarthur/list/manhattan-haunts',\n",
       "      'canonicalUrl': 'https://foursquare.com/becca_mcarthur/list/manhattan-haunts',\n",
       "      'createdAt': 1336094672,\n",
       "      'updatedAt': 1380845377,\n",
       "      'photo': {'id': '4e8cc9461081e3b3544e12e5',\n",
       "       'createdAt': 1317849414,\n",
       "       'prefix': 'https://fastly.4sqi.net/img/general/',\n",
       "       'suffix': '/0NLVU2HC1JF4DXIMKWUFW3QBUT31DC11EFNYYHMJG3NDWAPS.jpg',\n",
       "       'width': 492,\n",
       "       'height': 330,\n",
       "       'user': {'id': '742542',\n",
       "        'firstName': 'Time Out New York',\n",
       "        'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "         'suffix': '/XXHKCBSQHBORZNSR.jpg'},\n",
       "        'type': 'page'},\n",
       "       'visibility': 'public'},\n",
       "      'followers': {'count': 22},\n",
       "      'listItems': {'count': 187,\n",
       "       'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n",
       "         'createdAt': 1342934485}]}},\n",
       "     {'id': '4fae817be4b085f6b2a74d19',\n",
       "      'name': 'USA NYC MAN FiDi',\n",
       "      'description': 'Where to go for decent eats in the restaurant wasteland of Downtown NYC aka FiDi, along with Tribeca & Battery Park City.',\n",
       "      'type': 'others',\n",
       "      'user': {'id': '12113441',\n",
       "       'firstName': 'Kino',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/12113441-K5HTHFLU2MUCM0CM.jpg'}},\n",
       "      'editable': False,\n",
       "      'public': True,\n",
       "      'collaborative': False,\n",
       "      'url': '/kinosfault/list/usa-nyc-man-fidi',\n",
       "      'canonicalUrl': 'https://foursquare.com/kinosfault/list/usa-nyc-man-fidi',\n",
       "      'createdAt': 1336836475,\n",
       "      'updatedAt': 1556754919,\n",
       "      'photo': {'id': '55984992498e13ba75e353bb',\n",
       "       'createdAt': 1436043666,\n",
       "       'prefix': 'https://fastly.4sqi.net/img/general/',\n",
       "       'suffix': '/12113441_iOa6Uh-Xi8bhj2-gpzkkw8MKiAIs7RmOcz_RM7m8ink.jpg',\n",
       "       'width': 540,\n",
       "       'height': 960,\n",
       "       'user': {'id': '12113441',\n",
       "        'firstName': 'Kino',\n",
       "        'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "         'suffix': '/12113441-K5HTHFLU2MUCM0CM.jpg'}},\n",
       "       'visibility': 'public'},\n",
       "      'followers': {'count': 20},\n",
       "      'listItems': {'count': 273,\n",
       "       'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n",
       "         'createdAt': 1373909433}]}},\n",
       "     {'id': '4fddeff0e4b0e078037ac0d3',\n",
       "      'name': 'NYC Resturants',\n",
       "      'description': '',\n",
       "      'type': 'others',\n",
       "      'user': {'id': '21563126',\n",
       "       'firstName': 'Richard',\n",
       "       'lastName': 'R',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/21563126_v05J1KPw_SVj6Ehq9g8B9jeAGjFUMsU5QGl-NZ8inUQ7pKQm5bKplW37EmR7jS2A7GYPBBAtl.jpg'}},\n",
       "      'editable': False,\n",
       "      'public': True,\n",
       "      'collaborative': True,\n",
       "      'url': '/rickr7/list/nyc-resturants',\n",
       "      'canonicalUrl': 'https://foursquare.com/rickr7/list/nyc-resturants',\n",
       "      'createdAt': 1339944944,\n",
       "      'updatedAt': 1582232663,\n",
       "      'photo': {'id': '5072dd13e4b09145cdf782d1',\n",
       "       'createdAt': 1349704979,\n",
       "       'prefix': 'https://fastly.4sqi.net/img/general/',\n",
       "       'suffix': '/208205_fGh2OuAZ9qJ4agbAA5wMVNOSIm9kNUlRtNwj1N-adqg.jpg',\n",
       "       'width': 800,\n",
       "       'height': 800,\n",
       "       'user': {'id': '208205',\n",
       "        'firstName': 'Thalia',\n",
       "        'lastName': 'K',\n",
       "        'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "         'suffix': '/SNOOLCAW2AG04ZKD.jpg'}},\n",
       "       'visibility': 'public'},\n",
       "      'followers': {'count': 12},\n",
       "      'listItems': {'count': 194,\n",
       "       'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n",
       "         'createdAt': 1581655865}]}},\n",
       "     {'id': '5266c68a498e7c667807fe09',\n",
       "      'name': 'Foodie Love in NY - 02',\n",
       "      'description': '',\n",
       "      'type': 'others',\n",
       "      'user': {'id': '547977',\n",
       "       'firstName': 'WiLL',\n",
       "       'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "        'suffix': '/-Q5NYGDMFDMOITQRR.jpg'}},\n",
       "      'editable': False,\n",
       "      'public': True,\n",
       "      'collaborative': False,\n",
       "      'url': '/sweetiewill/list/foodie-love-in-ny--02',\n",
       "      'canonicalUrl': 'https://foursquare.com/sweetiewill/list/foodie-love-in-ny--02',\n",
       "      'createdAt': 1382467210,\n",
       "      'updatedAt': 1391995585,\n",
       "      'followers': {'count': 7},\n",
       "      'listItems': {'count': 200,\n",
       "       'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n",
       "         'createdAt': 1386809936}]}}]}]},\n",
       " 'hours': {'status': 'Closed until 11:30 AM',\n",
       "  'richStatus': {'entities': [], 'text': 'Closed until 11:30 AM'},\n",
       "  'isOpen': False,\n",
       "  'isLocalHoliday': False,\n",
       "  'dayData': [],\n",
       "  'timeframes': [{'days': 'Mon–Wed, Sun',\n",
       "    'includesToday': True,\n",
       "    'open': [{'renderedTime': '11:30 AM–11:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Thu–Sat',\n",
       "    'open': [{'renderedTime': '11:30 AM–Midnight'}],\n",
       "    'segments': []}]},\n",
       " 'popular': {'isOpen': False,\n",
       "  'isLocalHoliday': False,\n",
       "  'timeframes': [{'days': 'Today',\n",
       "    'includesToday': True,\n",
       "    'open': [{'renderedTime': 'Noon–2:00 PM'},\n",
       "     {'renderedTime': '5:00 PM–10:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Thu',\n",
       "    'open': [{'renderedTime': 'Noon–2:00 PM'},\n",
       "     {'renderedTime': '5:00 PM–10:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Fri',\n",
       "    'open': [{'renderedTime': 'Noon–3:00 PM'},\n",
       "     {'renderedTime': '5:00 PM–11:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Sat',\n",
       "    'open': [{'renderedTime': 'Noon–11:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Sun',\n",
       "    'open': [{'renderedTime': 'Noon–3:00 PM'},\n",
       "     {'renderedTime': '5:00 PM–8:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Mon',\n",
       "    'open': [{'renderedTime': 'Noon–2:00 PM'},\n",
       "     {'renderedTime': '6:00 PM–8:00 PM'}],\n",
       "    'segments': []},\n",
       "   {'days': 'Tue',\n",
       "    'open': [{'renderedTime': 'Noon–2:00 PM'},\n",
       "     {'renderedTime': '5:00 PM–10:00 PM'}],\n",
       "    'segments': []}]},\n",
       " 'pageUpdates': {'count': 0, 'items': []},\n",
       " 'inbox': {'count': 0, 'items': []},\n",
       " 'attributes': {'groups': [{'type': 'price',\n",
       "    'name': 'Price',\n",
       "    'summary': '$$',\n",
       "    'count': 1,\n",
       "    'items': [{'displayName': 'Price', 'displayValue': '$$', 'priceTier': 2}]},\n",
       "   {'type': 'payments',\n",
       "    'name': 'Credit Cards',\n",
       "    'summary': 'Credit Cards',\n",
       "    'count': 7,\n",
       "    'items': [{'displayName': 'Credit Cards',\n",
       "      'displayValue': 'Yes (incl. American Express)'}]},\n",
       "   {'type': 'outdoorSeating',\n",
       "    'name': 'Outdoor Seating',\n",
       "    'summary': 'Outdoor Seating',\n",
       "    'count': 1,\n",
       "    'items': [{'displayName': 'Outdoor Seating', 'displayValue': 'Yes'}]},\n",
       "   {'type': 'serves',\n",
       "    'name': 'Menus',\n",
       "    'summary': 'Happy Hour, Brunch & more',\n",
       "    'count': 8,\n",
       "    'items': [{'displayName': 'Brunch', 'displayValue': 'Brunch'},\n",
       "     {'displayName': 'Lunch', 'displayValue': 'Lunch'},\n",
       "     {'displayName': 'Dinner', 'displayValue': 'Dinner'},\n",
       "     {'displayName': 'Happy Hour', 'displayValue': 'Happy Hour'}]},\n",
       "   {'type': 'drinks',\n",
       "    'name': 'Drinks',\n",
       "    'summary': 'Beer, Wine & Cocktails',\n",
       "    'count': 5,\n",
       "    'items': [{'displayName': 'Beer', 'displayValue': 'Beer'},\n",
       "     {'displayName': 'Wine', 'displayValue': 'Wine'},\n",
       "     {'displayName': 'Cocktails', 'displayValue': 'Cocktails'}]},\n",
       "   {'type': 'diningOptions',\n",
       "    'name': 'Dining Options',\n",
       "    'summary': 'Delivery',\n",
       "    'count': 5,\n",
       "    'items': [{'displayName': 'Delivery', 'displayValue': 'Delivery'}]}]},\n",
       " 'bestPhoto': {'id': '4fad980de4b091b4626c3633',\n",
       "  'createdAt': 1336776717,\n",
       "  'source': {'name': 'Foursquare for Android',\n",
       "   'url': 'https://foursquare.com/download/#/android'},\n",
       "  'prefix': 'https://fastly.4sqi.net/img/general/',\n",
       "  'suffix': '/ya1iQFI7pLjuIJp1PGDKlrZS3OJdHCF7tpILMmjv_2w.jpg',\n",
       "  'width': 480,\n",
       "  'height': 640,\n",
       "  'visibility': 'public'},\n",
       " 'colors': {'highlightColor': {'photoId': '4fad980de4b091b4626c3633',\n",
       "   'value': -13619152},\n",
       "  'highlightTextColor': {'photoId': '4fad980de4b091b4626c3633', 'value': -1},\n",
       "  'algoVersion': 3}}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = requests.get(url).json()\n",
    "print(result['response']['venue'].keys())\n",
    "result['response']['venue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### B. Get the venue's overall rating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.0\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    print(result['response']['venue']['rating'])\n",
    "except:\n",
    "    print('This venue has not been rated yet.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "That is not a very good rating. Let's check the rating of the second closest Italian restaurant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This venue has not been rated yet.\n"
     ]
    }
   ],
   "source": [
    "venue_id = '4f3232e219836c91c7bfde94' # ID of Conca Cucina Italian Restaurant\n",
    "url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n",
    "\n",
    "result = requests.get(url).json()\n",
    "try:\n",
    "    print(result['response']['venue']['rating'])\n",
    "except:\n",
    "    print('This venue has not been rated yet.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Since this restaurant has no ratings, let's check the third restaurant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.4\n"
     ]
    }
   ],
   "source": [
    "venue_id = '3fd66200f964a520f4e41ee3' # ID of Ecco\n",
    "url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n",
    "\n",
    "result = requests.get(url).json()\n",
    "try:\n",
    "    print(result['response']['venue']['rating'])\n",
    "except:\n",
    "    print('This venue has not been rated yet.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Since this restaurant has a slightly better rating, let's explore it further."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### C. Get the number of tips"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result['response']['venue']['tips']['count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### D. Get the venue's tips\n",
    "> `https://api.foursquare.com/v2/venues/`**VENUE_ID**`/tips?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**`&limit=`**LIMIT**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Create URL and send GET request. Make sure to set limit to get all tips"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'meta': {'code': 200, 'requestId': '5e687489006dce001bbb6db3'},\n",
       " 'response': {'tips': {'count': 19,\n",
       "   'items': [{'id': '5ab1cb46c9a517174651d3fe',\n",
       "     'createdAt': 1521601350,\n",
       "     'text': 'A+ Italian food! Trust me on this: my mom’s side of the family is 100% Italian. I was born and bred to know good pasta when I see it, and Ecco is one of my all-time NYC favorites',\n",
       "     'type': 'user',\n",
       "     'canonicalUrl': 'https://foursquare.com/item/5ab1cb46c9a517174651d3fe',\n",
       "     'lang': 'en',\n",
       "     'likes': {'count': 0, 'groups': []},\n",
       "     'logView': True,\n",
       "     'agreeCount': 4,\n",
       "     'disagreeCount': 0,\n",
       "     'lastVoteText': 'Upvoted Dec 14, 2019',\n",
       "     'lastUpvoteTimestamp': 1576365433,\n",
       "     'todo': {'count': 0},\n",
       "     'user': {'id': '484542633',\n",
       "      'firstName': 'Nick',\n",
       "      'lastName': 'E',\n",
       "      'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n",
       "       'suffix': '/484542633_unymNUmw_FdPs3GjXHujmHcYnN4hf8kEPADlOZuIrdcdm97VX3tFqL7fFNMNA_8Gl9NlU1GYg.jpg'}},\n",
       "     'authorInteractionType': 'liked'}]}}}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Ecco Tips\n",
    "limit = 15 # set limit to be greater than or equal to the total number of tips\n",
    "url = 'https://api.foursquare.com/v2/venues/{}/tips?client_id={}&client_secret={}&v={}&limit={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION, limit)\n",
    "\n",
    "results = requests.get(url).json()\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Get tips and list of associated features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['id', 'createdAt', 'text', 'type', 'canonicalUrl', 'lang', 'likes', 'logView', 'agreeCount', 'disagreeCount', 'lastVoteText', 'lastUpvoteTimestamp', 'todo', 'user', 'authorInteractionType'])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tips = results['response']['tips']['items']\n",
    "\n",
    "tip = results['response']['tips']['items'][0]\n",
    "tip.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Format column width and display all tips"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['id', 'createdAt', 'text', 'type', 'canonicalUrl', 'lang', 'logView',\n",
      "       'agreeCount', 'disagreeCount', 'lastVoteText', 'lastUpvoteTimestamp',\n",
      "       'authorInteractionType', 'likes.count', 'likes.groups', 'todo.count',\n",
      "       'user.id', 'user.firstName', 'user.lastName', 'user.photo.prefix',\n",
      "       'user.photo.suffix'],\n",
      "      dtype='object')\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-59-d37b87450224>:1: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n",
      "  pd.set_option('display.max_colwidth', -1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0    0    A+ Italian food! Trust me on this: my mom’s side of the family is 100% Italian. I was born and bred to know good pasta when I see it, and Ecco is one of my all-time NYC favorites\n",
       "Name: text, dtype: object\n",
       "1    0    4\n",
       "Name: agreeCount, dtype: int64                                                                                                                                                                            \n",
       "2    0    0\n",
       "Name: disagreeCount, dtype: int64                                                                                                                                                                         \n",
       "3    0    5ab1cb46c9a517174651d3fe\n",
       "Name: id, dtype: object                                                                                                                                                            \n",
       "4    0    Nick\n",
       "Name: user.firstName, dtype: object                                                                                                                                                                    \n",
       "5    0    E\n",
       "Name: user.lastName, dtype: object                                                                                                                                                                        \n",
       "6    0    484542633\n",
       "Name: user.id, dtype: object                                                                                                                                                                      \n",
       "dtype: object"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.set_option('display.max_colwidth', -1)\n",
    "\n",
    "tips_df = pd.json_normalize(tips) # json normalize tips\n",
    "print(tips_df.columns)\n",
    "# columns to keep\n",
    "filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id', 'user.firstName', 'user.lastName', 'user.id']\n",
    "#filtered_columns\n",
    "tips_filtered = []\n",
    "for col in filtered_columns:\n",
    "    tips_filtered.append(tips_df.loc[:, col])\n",
    "\n",
    "# display tips\n",
    "tips_filtered = pd.Series(data = tips_filtered)\n",
    "tips_filtered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now remember that because we are using a personal developer account, then we can access only 2 of the restaurant's tips, instead of all 15 tips."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## 3. Search a Foursquare User\n",
    "> `https://api.foursquare.com/v2/users/`**USER_ID**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Define URL, send GET request and display features associated with user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{}\n"
     ]
    },
    {
     "ename": "KeyError",
     "evalue": "'user'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-66-77a09042e70d>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mrequests\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjson\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresults\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'response'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0muser_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mresults\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'response'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'user'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      9\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     10\u001b[0m \u001b[1;31m# display features associated with user\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyError\u001b[0m: 'user'"
     ]
    }
   ],
   "source": [
    "user_id = '484542633' # user ID with most agree counts and complete profile\n",
    "\n",
    "url = 'https://api.foursquare.com/v2/users/{}?client_id={}&client_secret={}&v={}'.format(user_id, CLIENT_ID, CLIENT_SECRET, VERSION) # define URL\n",
    "\n",
    "# send GET request\n",
    "results = requests.get(url).json()\n",
    "print(results['response'])\n",
    "user_data = results['response']['user']\n",
    "\n",
    "# display features associated with user\n",
    "user_data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "print('First Name: ' + user_data['firstName'])\n",
    "print('Last Name: ' + user_data['lastName'])\n",
    "print('Home City: ' + user_data['homeCity'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### How many tips has this user submitted?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "user_data['tips']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Wow! So it turns out that Nick is a very active Foursquare user, with more than 250 tips."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Get User's tips"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# define tips URL\n",
    "url = 'https://api.foursquare.com/v2/users/{}/tips?client_id={}&client_secret={}&v={}&limit={}'.format(user_id, CLIENT_ID, CLIENT_SECRET, VERSION, limit)\n",
    "\n",
    "# send GET request and get user's tips\n",
    "results = requests.get(url).json()\n",
    "tips = results['response']['tips']['items']\n",
    "\n",
    "# format column width\n",
    "pd.set_option('display.max_colwidth', -1)\n",
    "\n",
    "tips_df = json_normalize(tips)\n",
    "\n",
    "# filter columns\n",
    "filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id']\n",
    "tips_filtered = tips_df.loc[:, filtered_columns]\n",
    "\n",
    "# display user's tips\n",
    "tips_filtered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Let's get the venue for the tip with the greatest number of agree counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "tip_id = '5ab5575d73fe2516ad8f363b' # tip id\n",
    "\n",
    "# define URL\n",
    "url = 'http://api.foursquare.com/v2/tips/{}?client_id={}&client_secret={}&v={}'.format(tip_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n",
    "\n",
    "# send GET Request and examine results\n",
    "result = requests.get(url).json()\n",
    "print(result['response']['tip']['venue']['name'])\n",
    "print(result['response']['tip']['venue']['location'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Get User's friends"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "user_friends = json_normalize(user_data['friends']['groups'][0]['items'])\n",
    "user_friends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Interesting. Despite being very active, it turns out that Nick does not have any friends on Foursquare. This might definitely change in the future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Retrieve the User's Profile Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# 1. grab prefix of photo\n",
    "# 2. grab suffix of photo\n",
    "# 3. concatenate them using the image size  \n",
    "Image(url='https://igx.4sqi.net/img/user/300x300/484542633_mK2Yum7T_7Tn9fWpndidJsmw2Hof_6T5vJBKCHPLMK5OL-U5ZiJGj51iwBstcpDLYa3Zvhvis.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item4\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## 4. Explore a location\n",
    "> `https://api.foursquare.com/v2/venues/`**explore**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**`&limit=`**LIMIT**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### So, you just finished your gourmet dish at Ecco, and are just curious about the popular spots around the restaurant. In order to explore the area, let's start by getting the latitude and longitude values of Ecco Restaurant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "latitude = 40.715337\n",
    "longitude = -74.008848"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Define URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "url = 'https://api.foursquare.com/v2/venues/explore?client_id={}&client_secret={}&ll={},{}&v={}&radius={}&limit={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION, radius, LIMIT)\n",
    "url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Send GET request and examine results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "results = requests.get(url).json()\n",
    "'There are {} around Ecco restaurant.'.format(len(results['response']['groups'][0]['items']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Get relevant part of JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "items = results['response']['groups'][0]['items']\n",
    "items[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Process JSON and convert it to a clean dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "dataframe = json_normalize(items) # flatten JSON\n",
    "\n",
    "# filter columns\n",
    "filtered_columns = ['venue.name', 'venue.categories'] + [col for col in dataframe.columns if col.startswith('venue.location.')] + ['venue.id']\n",
    "dataframe_filtered = dataframe.loc[:, filtered_columns]\n",
    "\n",
    "# filter the category for each row\n",
    "dataframe_filtered['venue.categories'] = dataframe_filtered.apply(get_category_type, axis=1)\n",
    "\n",
    "# clean columns\n",
    "dataframe_filtered.columns = [col.split('.')[-1] for col in dataframe_filtered.columns]\n",
    "\n",
    "dataframe_filtered.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Let's visualize these items on the map around our location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "venues_map = folium.Map(location=[latitude, longitude], zoom_start=15) # generate map centred around Ecco\n",
    "\n",
    "\n",
    "# add Ecco as a red circle mark\n",
    "folium.features.CircleMarker(\n",
    "    [latitude, longitude],\n",
    "    radius=10,\n",
    "    popup='Ecco',\n",
    "    fill=True,\n",
    "    color='red',\n",
    "    fill_color='red',\n",
    "    fill_opacity=0.6\n",
    "    ).add_to(venues_map)\n",
    "\n",
    "\n",
    "# add popular spots to the map as blue circle markers\n",
    "for lat, lng, label in zip(dataframe_filtered.lat, dataframe_filtered.lng, dataframe_filtered.categories):\n",
    "    folium.features.CircleMarker(\n",
    "        [lat, lng],\n",
    "        radius=5,\n",
    "        popup=label,\n",
    "        fill=True,\n",
    "        color='blue',\n",
    "        fill_color='blue',\n",
    "        fill_opacity=0.6\n",
    "        ).add_to(venues_map)\n",
    "\n",
    "# display map\n",
    "venues_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item5\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## 5. Explore Trending Venues\n",
    "> `https://api.foursquare.com/v2/venues/`**trending**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Now, instead of simply exploring the area around Ecco, you are interested in knowing the venues that are trending at the time you are done with your lunch, meaning the places with the highest foot traffic. So let's do that and get the trending venues around Ecco."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# define URL\n",
    "url = 'https://api.foursquare.com/v2/venues/trending?client_id={}&client_secret={}&ll={},{}&v={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION)\n",
    "\n",
    "# send GET request and get trending venues\n",
    "results = requests.get(url).json()\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Check if any venues are trending at this time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "if len(results['response']['venues']) == 0:\n",
    "    trending_venues_df = 'No trending venues are available at the moment!'\n",
    "    \n",
    "else:\n",
    "    trending_venues = results['response']['venues']\n",
    "    trending_venues_df = json_normalize(trending_venues)\n",
    "\n",
    "    # filter columns\n",
    "    columns_filtered = ['name', 'categories'] + ['location.distance', 'location.city', 'location.postalCode', 'location.state', 'location.country', 'location.lat', 'location.lng']\n",
    "    trending_venues_df = trending_venues_df.loc[:, columns_filtered]\n",
    "\n",
    "    # filter the category for each row\n",
    "    trending_venues_df['categories'] = trending_venues_df.apply(get_category_type, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# display trending venues\n",
    "trending_venues_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, depending on when you run the above code, you might get different venues since the venues with the highest foot traffic are fetched live. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Visualize trending venues"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "if len(results['response']['venues']) == 0:\n",
    "    venues_map = 'Cannot generate visual as no trending venues are available at the moment!'\n",
    "\n",
    "else:\n",
    "    venues_map = folium.Map(location=[latitude, longitude], zoom_start=15) # generate map centred around Ecco\n",
    "\n",
    "\n",
    "    # add Ecco as a red circle mark\n",
    "    folium.features.CircleMarker(\n",
    "        [latitude, longitude],\n",
    "        radius=10,\n",
    "        popup='Ecco',\n",
    "        fill=True,\n",
    "        color='red',\n",
    "        fill_color='red',\n",
    "        fill_opacity=0.6\n",
    "    ).add_to(venues_map)\n",
    "\n",
    "\n",
    "    # add the trending venues as blue circle markers\n",
    "    for lat, lng, label in zip(trending_venues_df['location.lat'], trending_venues_df['location.lng'], trending_venues_df['name']):\n",
    "        folium.features.CircleMarker(\n",
    "            [lat, lng],\n",
    "            radius=5,\n",
    "            poup=label,\n",
    "            fill=True,\n",
    "            color='blue',\n",
    "            fill_color='blue',\n",
    "            fill_opacity=0.6\n",
    "        ).add_to(venues_map)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": false,
    "deletable": true,
    "jupyter": {
     "outputs_hidden": false
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# display map\n",
    "venues_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a id=\"item6\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Thank you for completing this lab!\n",
    "\n",
    "This notebook was created by [Alex Aklson](https://www.linkedin.com/in/aklson/). I hope you found this lab interesting and educational. Feel free to contact me if you have any questions!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This notebook is part of a course on **Coursera** called *Applied Data Science Capstone*. If you accessed this notebook outside the course, you can take this course online by clicking [here](http://cocl.us/DP0701EN_Coursera_Week2_LAB1)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "deletable": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<hr>\n",
    "Copyright &copy; 2018 [Cognitive Class](https://cognitiveclass.ai/?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}