v0.4.0 - Adding support for Gluon imperative models

New Features:

Gluon imperative model support

1. Added support for serving Gluon based imperative models.

Docs, Improvements, Bug fixes

Docs:

• Added documentation on Gluon model services.
• Added alexnet in Gluon to model serving to examples.
• Added Character-level Convolutional Neural Networks in Gluon to model serving examples.

Improvements:

• Gluon base service class implementation. (@vdantu )
• Improved Docker image setup time by , layering docker images. (@vrakesh, #343 )
• Docker image now can auto-detect number of available CPUs. (@vrakesh, #360 )
• Added pylint support. (@vdantu)
• Use cu90mkl mxnet on cuda gpu machines by default. (@vrakesh, #390 )

Bug Fixes:

• Fixed an issue, where empty folder was created when invalid model path is specified. (@vrakesh, #320 )
• Docker images now do not allow multiple instances of MMS to run. (@vrakesh, #337 )
• fixed pypi summary issue. (@aaronmarkham, #378 )
• Fixed error propagation from custom service to MMS. (@vrakesh, #387 )
• Fixed documentation bugs. (@vrakesh, #401 , #402 )
• Fixed version reading issue in MMS. (@vrakesh, #395 )
• Fixed post process latencies being high due to inference variables being lazy evaluated. (@vrakesh, #414 )

v0.3.0

New Features:

New CLI to interact with MMS running in a container

1. New options to start/stop/restart MMS in container.
2. Option to point to different configuration files for each MMS run.
3. Multiple bug fixes.

Optimized and pre-configured MMS container images

1. Published the container image to Docker Hub.
2. The default configuration in these containers and the example configuration in the repository are optimized for CPU and GPU AWS EC2 instances.

Bug fixes and Docs

Docs:

• README documents.
• Added docs to depict orchestrating MMS as an AWS FARGATE service.
• Added docs for optimizing the MMS configuration for different EC2 instances.

Bug Fixes:

• Corrected Readme and advanced-settings doc for MMS container (@aaronmarkham )
• Documentation for optimised setup for GPU and CPU EC2 instances (@ankkhedia )
• Optimized MMS GPU-container to utilize all GPUs in an GPU instance (@ankkhedia )
• Documentation for launching MMS on AWS Fargate service
• Added integration tests framework (@ankkhedia )
• Doc update on Production usage. Describes why Container images are better for prod. (#336)
• Streamlining Container based MMS orchestration (@vdantu)
• Optimized the model file downloads for container runs of MMS. (@vdantu)
• Fixed bugs in mxnet-model-export (@ankkhedia )

v0.2.0

New features

ONNX model support

Model server now supports models stored in the Open Neural Network Exchange (ONNX) format. See Export an ONNX Model for details.

Cloudwatch metrics

Model server can publish host and model related metrics to Amazon Cloudwatch. See Cloudwatch metrics for details.

Improvements and bug fixes

• Fixing LatencyOverall unit reporting (@lupesko, #317)
• update onnx-mxnet (@jesterhazy, #316)
• remove docs images (@jesterhazy, #315)
• added cloudwatch metrics section (@aaronmarkham, #314)
• update docker scripts (@jesterhazy, #313)
• added toc, logos, and kitten image (@aaronmarkham, #311)
• add unit test for hyphenated model files (@jesterhazy, #308)
• Fix validate_prefix_match (@knjcode, #307)
• align metrics names/units with standard cloudwatch metrics (@jesterhazy, #303)
• Fix race condition when multiple gunicorn workers try to download same models (@yuruofeifei, #302)
• Fix epoch number validation (@knjcode, #298)
• remove License field (@jesterhazy, #297)
• update error messages for model export (@jesterhazy, #295)
• Fixing and updating docker setup (@lupesko, #294)
• public domain image examples for SSD outputs (@aaronmarkham, #291)
• refactored export info; added shortcuts and other improvements (@aaronmarkham, #290)
• added four onnx-exported models to zoo; added onnx support to server intro (@aaronmarkham, #288)
• Documentation updates for 0.2 (@aaronmarkham, #285)
• Improve cloudwatch integration, fix several issues. (@yuruofeifei, #283)
• fail fast when user tries to serve onnx model directly (@jesterhazy, #280)
• fix importlib warning (#254) (@jesterhazy, #279)
• bump version (@yuruofeifei, #250)
• Onnx and metrics docs (@yuruofeifei, #244)
• Add onnx support (@yuruofeifei, #240)
• Zoo updates (@aaronmarkham, #234)
• Zoo updates with details for each model (@aaronmarkham, #233)

Initial release of Model Server

Key capabilities of Model Server for Apache MXNet v0.1.5:

• Tooling to package and export all model artifacts into a single “model archive” file that encapsulates everything required for serving an MXNet model.
• Automated setup of a serving stack, including HTTP inference endpoints, MXNet-based engine, all automatically configured for the specific models being hosted.
• Pre-configured Docker images, setup with NGINX, MXNet and MMS, for scalable model serving.
• Ability to customize every step in the inference execution pipeline, from model initialization, through pre-processing and inference, and up to post-processing the model’s output.
• Real time operational metrics to monitor the inference service and endpoints, covering key metrics such as latencies, resource utilization and errors.
• OpenAPI-enabled service, that is easy to integrate with, and that can auto-generate client code for popular stacks such as Java, JavaScript, C# and more.