sagemaker-python-sdk/mxnet_gluon_sentiment/mxnet_sentiment_analysis_with_gluon.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis with Apache MXNet and Gluon\n",
    "\n",
    "This tutorial shows how to train and test a Sentiment Analysis (Text Classification) model on Amazon SageMaker using Apache MXNet and the Gluon API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download training and test data\n",
    "\n",
    "In this notebook, we train a Sentiment Analysis model on the [SST-2 (Stanford Sentiment Treebank 2) dataset](https://nlp.stanford.edu/sentiment/index.html). This dataset consists of movie reviews with one sentence per review. The task is to classify the review as either positive or negative.\n",
    "\n",
    "We download the preprocessed version of this dataset from the links below. Each line in the dataset has space separated tokens, with the first token being the label: 1 for positive and 0 for negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "mkdir data\n",
    "\n",
    "curl https://raw.githubusercontent.com/saurabh3949/Text-Classification-Datasets/master/stsa.binary.phrases.train > data/train\n",
    "curl https://raw.githubusercontent.com/saurabh3949/Text-Classification-Datasets/master/stsa.binary.test > data/test "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload the data\n",
    "\n",
    "We use the `sagemaker.s3.S3Uploader` to upload our datasets to an Amazon S3 location. The return value `inputs` identifies the location -- we use this later when we start the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import s3, session\n",
    "\n",
    "bucket = session.Session().default_bucket()\n",
    "inputs = s3.S3Uploader.upload('data', 's3://{}/mxnet-gluon-sentiment-example/data'.format(bucket))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement the training function\n",
    "\n",
    "We need to provide a training script that can run on the SageMaker platform. The training scripts are essentially the same as one you would write for local training, but you can also access useful properties about the training environment through various environment variables. In addition, hyperparameters are passed to the script as arguments. For more about writing an MXNet training script for use with SageMaker, see [the SageMaker documentation](https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#prepare-an-mxnet-training-script).\n",
    "\n",
    "The script here is a simplified implementation of [\"Bag of Tricks for Efficient Text Classification\"](https://arxiv.org/abs/1607.01759), as implemented by Facebook's [FastText](https://github.com/facebookresearch/fastText/) for text classification. The model maps each word to a vector and averages vectors of all the words in a sentence to form a hidden representation of the sentence, which is inputted to a softmax classification layer. For more details, please refer to [the paper](https://arxiv.org/abs/1607.01759).\n",
    "\n",
    "At the end of every epoch, our script also checks the validation accuracy, and checkpoints the best model so far, along with the optimizer state, in the folder `/opt/ml/checkpoints`. (If the folder `/opt/ml/checkpoints` does not exist, this checkpointing step is skipped.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize 'sentiment.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run a SageMaker training job\n",
    "\n",
    "The `MXNet` class allows us to run our training function on SageMaker infrastructure. We need to configure it with our training script, an IAM role, the number of training instances, and the training instance type. In this case we run our training job on a single `c4.2xlarge` instance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import get_execution_role\n",
    "from sagemaker.mxnet import MXNet\n",
    "\n",
    "m = MXNet('sentiment.py',\n",
    "          role=get_execution_role(),\n",
    "          train_instance_count=1,\n",
    "          train_instance_type='ml.c4.xlarge',\n",
    "          framework_version='1.6.0',\n",
    "          py_version='py3',\n",
    "          distributions={'parameter_server': {'enabled': True}},\n",
    "          hyperparameters={'batch-size': 8,\n",
    "                           'epochs': 2,\n",
    "                           'learning-rate': 0.01,\n",
    "                           'embedding-size': 50, \n",
    "                           'log-interval': 1000})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we've constructed our `MXNet` estimator, we can fit it using the data we uploaded to S3. SageMaker makes sure our data is available in the local filesystem, so our training script can simply read the data from disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m.fit(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen from the logs, our model gets over 80% accuracy on the test set using the above hyperparameters.\n",
    "\n",
    "After training, we use our `MXNet` object to build and deploy an `MXNetPredictor` object. This creates a SageMaker Endpoint that we can use to perform inference. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "predictor = m.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With our predictor, we can perform inference on a JSON-encoded string array. \n",
    "\n",
    "The predictor runs inference on our input data and returns the predicted sentiment (1 for positive and 0 for negative)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data = [\"this movie was extremely good .\",\n",
    "        \"the plot was very boring .\",\n",
    "        \"this film is so slick , superficial and trend-hoppy .\",\n",
    "        \"i just could not watch it till the end .\",\n",
    "        \"the movie was so enthralling !\"]\n",
    "\n",
    "response = predictor.predict(data)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup\n",
    "\n",
    "After you have finished with this example, remember to delete the prediction endpoint to release the instance(s) associated with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_mxnet_p36",
   "language": "python",
   "name": "conda_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 2
}