Skip to content

Latest commit

 

History

History
55 lines (36 loc) · 2.75 KB

README.md

File metadata and controls

55 lines (36 loc) · 2.75 KB
Raw

iMNIST

iMNIST is a iOS app that recognizes the digits you draw using a trained TensorFlow NN.

How to intall it?

Requirements :

  • Xcode 8
  • A device or simulator under iOS 10.
  • Accelerate is needed for that app. We need one of its subset called BNNS. Make sure it is imported in Targets -> iMNIST -> General -> Linked frameworks and libraries.

Clone the repo, open it in Xcode and sign it with your own developer profile.

Nota bene : The code was written following the Swift 3 syntax.

How does it work ?

UI

Here is a capture from the app :

  • The user draws a figure between the two lines.
  • He presses the compute button and the app will try to guess what figure was drawn.
  • The drawn figure disappears.

In-depth

To achieve this simple functionnality the following steps are necessary.

Prior to app launch :

  • Export the weights and biases from the MNIST simple model from TensorFlow website. They appear in the app Bundle as "weights.data" and "bias.data".

The user draws on the screen.

  • Convert the user drawing to an UIImage.
  • Create a 28*28 grayscale representation of that UIImage (to be compliant with MNIST requirements).
  • Convert that UIImage to a normalized vector representation of the pixels values (they must be between 0 and 1).
  • Import the data from "weights.data" and "bias.data" and parse it.
  • Build a BNNS model from the biases, the weights and the input data.
  • Apply the BNNS filter gives us a vector of 10 real values.
  • Apply Softmax to the ouput vector gives us a normalized version of that vector.
  • Output the highest value and show the guessed figure to the user.

Where to go from here ?

This project was made in an attempt to do inference from a TensorFlow model on an iOS device using Apple's BNNS. The motivation for using it: BNNS processing is optimized for iOS devices CPUs.
There is no back-propagation with BNNS though, you'll need to train the model on a server and import the weights and biases. A pipeline with push notifications to download the new model seems relevant.

Other solutions exists to do this on iOS: