epic: dictionary-based word-breakers 🔬

[mcdurdin]

No description provided.

[keymanapp-test-bot]

User Test Results

Test specification and instructions

ERROR: user tests have not yet been defined

Test Artifacts

• Android
  • Keyman for Android apk
  • FirstVoices for Android 16.0 apk
  • FirstVoices Keyboards for Android apk
  • KeyboardHarness apk
  • Keyman for Android 16.0 apk
  • KMSample1 apk
  • KMSample2 apk
• Developer
  • Keyman Developer - build pending
  • Compiler Regression Tests - build pending
  • kmcomp.zip - build pending
• iOS
  • Keyman for iOS (simulator image)
  • FirstVoices Keyboards for iOS (simulator image)
  • TestFlight internal PR build version - 18.0.138 (0.12142.12259)
• Keyboards
  • Test Keyboards - build pending
• Web
  • KeymanWeb Test Home
• Windows
  • Keyman for Windows
  • FirstVoices Keyboards for Windows
  • Text Editor (32 bit)
  • Text Editor (64 bit)

[jahorton]

I got to wondering if there are any "relatively simple" ways to avoid spinning up a WebView to run the model-compiler, should we decide to keep the user-dictionary compilation completely separate from the keyboard.

After a bit of searching, I found this: https://github.com/nodejs-mobile - a library for running Node-oriented JS scripts for mobile devices. That said, it'd be a new dependency.

[jahorton]

Other notable thoughts:

We should probably not associate a language code with user-dictionary data. That is, we collate the data once and use that with any language supporting predictive-text.

My original strategy (as of #11994) was to blend the models into a single, "traversable" model.

• This would require that the standard lexical model for each language implements the LexiconTraversal interface, though - which is not strictly required for all custom models.
  • We'd need an alternate strategy to support scenarios where a language-specific custom model lacks this feature.
• Thinking ahead, we'd want a similar strategy to be in place once we start doing 'learning', which would adjust a model's probability data to better suit the user's actual typing patterns.

@mcdurdin previously suggested instead doing multiple correction-searches and picking the best from among their results after applying relative weighting. This would work, though it would also require support for multiple correction searches that does not yet exist.

• They should likely use the same allotment for total execution time... likely requiring some form of load balancing.
• We'd likely need the ability to pause and resume whichever search is currently returning 'more likely' paths at the time.

[mcdurdin]

• we do have to solve the issue of data transfer in one manner or other.

Data transfer into the webview could be via local file: or http: request. This opens up a number of extensibility questions for KeymanWeb itself and how we could make the web-based experience consistent with the Keyman Android/iOS app experience.