feat(developer): kmc generate #11014

mcdurdin · 2024-03-16T21:38:22Z

kmc-generate is a tool that will replace most aspects of kmconvert (Windows-only). The remaining functionality in kmconvert is importing keyboards from Windows DLLs, which currently requires a running Windows environment (anticipate replacement of this last piece of functionality with something in epic/kmc-convert (#12191))..

kmc-generate follows the same patterns as other kmc modules. All functions are accessible from the kmc command-line tool, and also scriptable with the standalone npm module.

Usage:

kmc generate keyman-keyboard my_kmn_keyboard [options]
kmc generate ldml-keyboard my_ldml_keyboard [options]
kmc generate lexical-model my.en.lexical_model_uniq [options]

Options vary from generator-to-generator. At this point, the generation tooling is largely template-based, which does limit a little what this tool can do. Renaming keyboards and other functionality will emerge in a future kmc module.

TODO

Add KB article on migrating from KMFL to Keyman for Linux help.keyman.com#1335 -- update kmconvert ref
additional validation of ids for keyboard and model filenames (e.g. casing)

User Testing

The purpose of the test here is to verify that the generate command creates a template project for a keyboard or lexical model with appropriate basic metadata.

For each command, run kmc generate <command> --help, to list options for generating the project. Include as many of the options as possible in your next command to generate a template project with them, and give it a sensible name e.g. generated_project.

TEST_KEYMAN_KEYBOARD: For the keyman-keyboard command, follow the instructions above to generate a basic Keyman keyboard. Then use kmc build generated_project to verify that the keyboard builds. Install the resulting .kmp and verify that it installs without error. View the source of the files, and verify that the contents of the generated files look correct and match the parameters you provided.

TEST_LDML_KEYBOARD: For the ldml-keyboard command, follow the instructions above to generate a basic LDML .xml keyboard (e.g. using id generated_ldml_keyboard). Then use kmc build generated_ldml_keyboard to verify that the keyboard builds. Install the resulting .kmp and verify that it installs without error. View the source of the files, and verify that the contents of the generated files look correct and match the parameters you provided.

TEST_LEXICAL_MODEL: For the lexical-model command, follow the instructions above to generate a basic lexical model (e.g. using id generated.en.model). Then use kmc build generated.en.model to verify that the lexical model builds. Install the resulting .kmp on an Android phone and verify that it installs without error. View the source of the files, and verify that the contents of the generated files look correct and match the parameters you provided.

keymanapp-test-bot · 2024-03-16T21:38:26Z

User Test Results

Test specification and instructions

✅ TEST_KEYMAN_KEYBOARD (PASSED) (notes)
✅ TEST_LDML_KEYBOARD (PASSED) (notes)
✅ TEST_LEXICAL_MODEL (PASSED) (notes)

Test Artifacts

Android
Developer
iOS
- Keyman for iOS (simulator image)
- FirstVoices Keyboards for iOS (simulator image)
- TestFlight internal PR build version - 18.0.118 (0.11014.12090)
Keyboards
- Test Keyboards
Web
- KeymanWeb Test Home
Windows

Plenty of todo items left but this is a mostly-functional module for the kmc generate command for automatically creating keyboard and model projects.

Cleanup various todo items for kmc generate. Outstanding TODO item is icon generation for .kmn.

Note: these are more e2e than unit tests. It may be difficult to convert them into solid unit tests due to the way that kmc-generate works based on templates.

mcdurdin · 2024-09-16T23:01:19Z

@dinakaranr encountered missing files in user test.

C:\Program Files (x86)\Keyman\Keyman Developer>kmc generate keyman-keyboard my_kmn_KeyMan_keyboard
fatal KM05001: Unexpected exception: Error: source file C:\Program Files (x86)\Keyman\Keyman Developer\kmc\template\kmn-keyboard\source\welcome.htm does not exist
Call stack:
Error: source file C:\Program Files (x86)\Keyman\Keyman Developer\kmc\template\kmn-keyboard\source\welcome.htm does not exist
    at _KeymanKeyboardGenerator.transform (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:442936:13)
    at file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:442930:83
    at Array.every (<anonymous>)
    at _KeymanKeyboardGenerator.transformAll (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:442930:63)
    at _KeymanKeyboardGenerator.generate (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:442914:17)
    at _KeymanKeyboardGenerator.run (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:443020:15)    at doGenerate (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:443802:37)
    at async generate (file:///C:/Program%20Files%20(x86)/Keyman/Keyman%20Developer/kmc/kmc.mjs:443787:8)
    This error has been automatically reported to the Keyman team.
      Identifier:  5582bbdefce54184b38f3689a801ae5d
      Application: Keyman Developer
      Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/5582bbdefce54184b38f3689a801ae5d/
C:\Program Files (x86)\Keyman\Keyman Developer>

mcdurdin · 2024-09-26T23:49:49Z

Projects are constructed automatically now. Some issues still with default runs:

LDML keyboard issues

$ kmc generate ldml-keyboard woo
$ kmc build woo
woo.kpj - info KM05002: Building woo/woo.kpj
woo.xml - info KM05002: Building woo/source/woo.xml
woo.xml - info KM05006: woo/source/woo.xml built successfully.
woo.kps - info KM05002: Building woo/source/woo.kps
woo.kps - info KM05006: woo/source/woo.kps built successfully.
woo.kpj - info KM05002: Building woo/woo.kpj
woo.kpj - error KM09010: The Info.Description field in the package C:\Users\marc\woo\source\woo.kps is required, but is missing or empty.
woo.kpj - info KM05007: woo/woo.kpj failed to build.
woo.kpj - info KM0500C: Project woo/woo.kpj failed to build.

Once I added a description, the project built successfully.

Keyman keyboard issues

$ kmc generate keyman-keyboard kk
$ kmc build kk
kk.kpj - info KM05002: Building kk/kk.kpj
kk.kmn - info KM05002: Building kk/source/kk.kmn
kk.kmn:8 - error KM0207B: At least one compile target must be specified
kk.kmn - info KM05007: kk/source/kk.kmn failed to build.
kk.kpj - info KM0500C: Project kk/kk.kpj failed to build.

I manually fixed the missing targets (store(&targets) 'any'), and then the following error arose:

$ kmc build kk
kk.kps - error KM04008: Keyboard kk was listed in <Keyboards> but a corresponding .kmx file was not found in <Files>

Added <File><Name>../build/kk.kmx</Name></File>, and then received:

kk.kpj - error KM09010: The Info.Description field in the package C:\Users\marc\Projects\kk\source\kk.kps is required, but is missing or empty.

Finally, added a description, got this (we should generate a touch layout by default):

kk.kpj - hint KM0401C: The keyboard kk has been included for touch platforms, but does not include a touch layout.

lexical model compiler issues

Model compiler build succeeded.

other issues

Secondary issue: kmc does not work as a command in git bash; needs kmc.cmd. Should include both filenames in Keyman Developer install.

dinakaranr · 2024-09-30T10:41:53Z

Test Results

I tested this issue with the attached "Keyman 18.0.118-alpha-test-11014" build on Windows 10 and 11. I'm sharing my observation here.
Created a new folder under the download folder.
Open the command prompt in this folder.
Executed below commands to generate keyman keyboards.

kmc generate keyman-keyboard kmnkeyboard
kmc generate ldml-keyboard ldml_keyboard
kmc generate lexical-model en.lexicalmodeluniq

Verified that the keyboard folder was created with the specified name

TEST_KEYMAN_KEYBOARD (Passed):

Go to the folder that had keyboards.
Open the command prompt in this folder.
Execute the "kmc build keyboard_folder_name" command.
Verified that the build folder was created inside the keyboard's folder.
Verified that the *.kmp file was created inside the build folder.
Install the *.kmp file in Windows 10 or 11
Verified that tha keyboard was added under "English"
Go to the source folder.
Verified that the generated files.

TEST_LDML_KEYBOARD (Passed):

Go to the folder that had keyboards.
Open the command prompt in this folder.
Execute the "kmc build keyboard_folder_name" command.
Verified that the build folder was created inside the keyboard's folder.
Verified that the *.kmp file was created inside the build folder.
Install the *.kmp file in Windows 10 or 11
Verified that the keyboard was added under "English"
Go to the source folder.
Verified that the generated files.

TEST_LEXICAL_MODEL (Passed):

Go to the folder that had keyboards.
Open the command prompt in this folder.
Execute the "kmc build keyboard_folder_name" command.
Verified that the build folder was created inside the keyboard's folder.
Verified that the *.kmp file was created inside the build folder.
Install the *.kmp file on an Android 12 mobile device.
Verified that the keyboard was added under "English."
Go to the source folder.
Verified that the generated files.

keymanapp-test-bot bot added the user-test-missing User tests have not yet been defined for the PR label Mar 16, 2024

keymanapp-test-bot bot added this to the B17S4 milestone Mar 16, 2024

mcdurdin force-pushed the feat/developer/kmc-generate branch 2 times, most recently from 3d22452 to 027ebfa Compare March 17, 2024 00:52

github-actions bot added developer/ common/ developer/compilers/ common/web/ feat labels Mar 17, 2024

mcdurdin modified the milestones: B17S4, B17S5 Mar 30, 2024

darcywong00 modified the milestones: B17S5, B17S6 Apr 12, 2024

darcywong00 modified the milestones: B17S6, A18S1 Apr 28, 2024

darcywong00 modified the milestones: A18S1, A18S2 May 11, 2024

mcdurdin modified the milestones: A18S2, A18S3 May 24, 2024

mcdurdin modified the milestones: A18S3, A18S4 Jun 7, 2024

darcywong00 modified the milestones: A18S4, A18S5 Jun 21, 2024

mcdurdin added 4 commits June 26, 2024 06:45

feat(developer): kmc-generate module

1ab9574

Plenty of todo items left but this is a mostly-functional module for the kmc generate command for automatically creating keyboard and model projects.

feat(developer): kmc-generate docs and tests started

9003dbb

feat(developer): kmc generate cleanup

125cfe6

Cleanup various todo items for kmc generate. Outstanding TODO item is icon generation for .kmn.

feat(developer): unit tests for kmc-generate

a45abe4

Note: these are more e2e than unit tests. It may be difficult to convert them into solid unit tests due to the way that kmc-generate works based on templates.

mcdurdin force-pushed the feat/developer/kmc-generate branch from 5ccabea to a45abe4 Compare June 25, 2024 20:47

chore(developer): fixup tests and scripts for kmc-generate

413ae60

github-actions bot added common/ common/web/ labels Sep 9, 2024

mcdurdin added 4 commits September 10, 2024 10:54

test(developer): add unit test for generate module

4a2e6fd

chore(developer): generate no longer needs to generate .model_info

021d434

chore(developer): add api-extractor for kmc-generate

edc5c93

chore(developer): use new npm publish for kmc-generate

178325c

keymanapp-test-bot bot added has-user-test user-test-required User tests have not been completed and removed user-test-missing User tests have not yet been defined for the PR labels Sep 12, 2024

mcdurdin marked this pull request as ready for review September 12, 2024 06:18

mcdurdin requested review from darcywong00 and jahorton as code owners September 12, 2024 06:18

Merge branch 'master' into feat/developer/kmc-generate

8424cb9

mcdurdin linked an issue Sep 13, 2024 that may be closed by this pull request

feat(developer): convert kmconvert (Delphi) to node, kmc generate #10509

Open

darcywong00 modified the milestones: A18S10, A18S11 Sep 14, 2024

mcdurdin marked this pull request as draft September 16, 2024 23:00

mcdurdin added 4 commits September 17, 2024 17:05

fix(developer): copy templates into bundle

48d4e65

feat(developer): add .gitignore to kmc generate templates

ec6c26c

Merge branch 'master' into feat/developer/kmc-generate

10e4bba

chore(developer): update eslint config path

025b9ed

mcdurdin added 2 commits September 26, 2024 17:12

chore(developer): cleanup targets and generated files in kmc-generate

5886847

chore(developer): fix kmc-generate test fixtures

0997986

mcdurdin marked this pull request as ready for review September 27, 2024 00:34

darcywong00 modified the milestones: A18S11, A18S12 Sep 28, 2024

keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Sep 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(developer): kmc generate #11014

feat(developer): kmc generate #11014

mcdurdin commented Mar 16, 2024 •

edited

Loading

keymanapp-test-bot bot commented Mar 16, 2024 •

edited

Loading

mcdurdin commented Sep 16, 2024 •

edited

Loading

mcdurdin commented Sep 26, 2024

dinakaranr commented Sep 30, 2024

feat(developer): kmc generate #11014

Are you sure you want to change the base?

feat(developer): kmc generate #11014

Conversation

mcdurdin commented Mar 16, 2024 • edited Loading

TODO

User Testing

keymanapp-test-bot bot commented Mar 16, 2024 • edited Loading

User Test Results

Test Artifacts

mcdurdin commented Sep 16, 2024 • edited Loading

mcdurdin commented Sep 26, 2024

LDML keyboard issues

Keyman keyboard issues

lexical model compiler issues

other issues

dinakaranr commented Sep 30, 2024

Test Results

mcdurdin commented Mar 16, 2024 •

edited

Loading

keymanapp-test-bot bot commented Mar 16, 2024 •

edited

Loading

mcdurdin commented Sep 16, 2024 •

edited

Loading