Skip to content

Use of Azure Cognitive Services' Immersive Reader in a Xamarin mobile application

License

Notifications You must be signed in to change notification settings

NickSpag/MobileImmersiveReader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MobileImmersiveReader

A Xamarin app using Azure Computer Vision's Read API and the Azure Immersive Reader's javascript sdk.

Background:

The Immersive Reader is an Azure Cognitive Service for developers who want to embed inclusive capabilities into their apps for enhancing text reading and comprehension for users regardless of age or ability. It's largely designed, from what it seems, to be added to web properties to easily extend their inclusive functionality. I wanted to try expanding the capabilities a little by integrating a "Take Photo" functionality.

App:

This is a Xamarin.iOS sample application. It uploads an image of a book page or text, either taken with the camera or picked from the gallery, to Azure Computer Vision to be read. The resulting text is built in to a makeshift html page that loads and launches the Javascript sdk for the Immersive Reader, which is pushed in to a WKWebView.

Since it's being demo'd on a phone with a retina screen and there aren't yet options to granularly control the UI of the reader, the controls are a bit small. I'd need to play with increasing the web view's zoom/scale/etc, as well as loading the reader with preset options if that functionality becomes available in the future. But it's a fun example in the meantime:

Using this app: You'll need an Azure subscription with an Immersive Reader resource, and a Cognitive Services resource. Once those are created, place the keys and endpoint urls in the respective string decelerations in the Help/PrivateKeys.cs file.

An animated gif of the app showing the gallery button being pressed, a photo of a book page being picked, an "Analyzing" notification, and ending with the immersive reader showing that book's text An animated gif of the app where the immersive reader's options are shown. Enables line focus item and a gray overlay on the reader obscures only a few lines. Enables the sylables option and the reader's words split in to sylables. Enables the noun highlighting and the nouns in the reader's page turn purple. Increases the text size and the reader's text gets substantially larger

License

Provided open source under MIT

About

Use of Azure Cognitive Services' Immersive Reader in a Xamarin mobile application

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages