Import LB3 model JSON file to LB4 model class

[bajtos]

As an LB3 user migrating my model definition files to LB4, I would like to call lb4 CLI to generate an LB4 model class from my LB3 model JSON file.

lb4 import-lb3-models legacy-lb3app/server/server.js
# load the LB3 app
# ask the user which models to import
# create src/models/product.ts etc.

The initial version should handle property definitions only.

For example, let's take an LB3 Product model with a name property:

{
  "name": "Product",
  "base": "PersistedModel",
  "properties": {
    "name": {
      "type": "string",
      "required": true
    }
  }
}

This LB3 definition should be translated into an LB4 TypeScript model class:

@model()
class Product extends Entity {
  @property({required: true})
  name: string;
}

Acceptance criteria

• A new CLI flag for lb4 model, or perhaps a new command, accepting a path to LB3 model file, that will scaffold (code-generate) an LB4 model file from LB3 definition. See Model Discovery #2245 for related prior work. (/cc @marvinirwin)
  --> see CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688
• The following base model classes should be recognized and mapped to LB4:
  • Model -> Model
  • PersistedModel -> Entity
  • KeyValueModel -> (to be determined)
    When the base model class is not recognized, the CLI should abort with a descriptive error.
    --> see CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688
• The following model settings should trigger a warning when a non-empty value is provided in LB3 model file:
  • relations
  • method
  • mixins
  • acls
    --> see CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688
• All other model settings should be copied as-is to LB4 model definition. IMPORTANT: settings can be specified either at top-level or inside options property, we need to support both flavors. Notable settings to test & support:
  • forceId
  • strict
  • connector-specific config like SQL table name and MongoDB collection name
    --> see CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688
• Documentation at CLI level
  --> see CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688
• A blog post
• Follow-up stories for things that were left out of the initial implementation. Most notably support for non-default base model classes and relations.

See #2036 for the next step that will allow developers to take LB4 model and expose it via REST with little to no configuration.

Open discussion points & todos

• Should we offer built-in models like User and AccessToken for importing?
• Should we select all models by default?
• How to automate model selection? Should we introduce a cli option like --models User,AccessToken? Use positional arguments like lb4 import-model my-lb3-app User AccessToken?
• Process TODO & FIXME comments in the initial implementation CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL] #3688 and convert them to sub-tasks or follow-up issues
• Write automated test to verify import of models attached to the following connectors:
  • SQL (MySQL or Postgres)
  • MongoDB (how is ObjectID type handled?)
  • Cloudant (how is _rev property handled?)

Out of scope

• relations
• remote methods
• ACLs
• mixins
• Custom model property settings must be preserved, e.g. SQL column names.
• A new section in the migration guide

[marioestradarosa]

@bajtos , currently the lb4 @model decorator can take the lb3 version model in json format as follows:

@model(
{
  "name": "Product",
  "properties": {
    "name": {
      "type": "string",
      "required": true
    }
  }
})
class Product extends Entity {
}

[emonddr]

How are we going to handle the model-config.json ?

[buokae]

Is there a migration manual somewhere? I have eight months left to migrate my project from lb3 to lb4 and no idea where to start.

[bajtos]

@buokae

Is there a migration manual somewhere? I have eight months left to migrate my project from lb3 to lb4 and no idea where to start.

Not yet, unfortunately. Please follow the discussion in #1849.

[buokae]

Alright, thank you.

[bajtos]

How are we going to handle the model-config.json ?

In LB3, model setup has two steps:

1. A model class is defined using data from the per-model JSON file.
2. A model is attached to a datasource and exposed via REST API, this is controlled by configuration in model-config.json.

The goal of this story is to provide LB4 version for the first step only.

The second step will be handled by #2036, see also the spike that have already begun: #3617

[bajtos]

The first draft of the implementation (early WIP): #3688

[bajtos]

Custom model property settings must be preserved, e.g. SQL column names.

I'll be opening a follow-up story for this point.

A new section in the migration guide

We don't have much of migration guide yet, let's make this item a part of #3718.

Should we offer built-in models like User and AccessToken for importing?

Let's offer built-in models for the initial version and then use feedback from users to decide what changes (if any) we want to make.

Should we select all models by default?

Since we cannot import models inheriting from custom app models yet, I feel selecting all models by default will not work for most developers. I am also not sure how many developers will want to import LB3 built-in models like User and AccessToken.

How to automate model selection? Should we introduce a cli option like --models User,AccessToken? Use positional arguments like lb4 import-model my-lb3-app User AccessToken?

Let's wait until there is user demand for this feature.

Write automated test to verify import of models attached to the following connectors:

• SQL (MySQL or Postgres)
• MongoDB (how is ObjectID type handled?)
• Cloudant (how is _rev property handled?)

I'll be opening a follow-up issue for MongoDB. Based on docs in our cloudant connector, _rev property has to be defined explicitly by users (app developers), therefore I don't expect any issues here. As for SQL, I am not aware of any potential issue either. Let's wait for user feedback before investing more effort into this area.

[bajtos]

Created the following follow-up stories:

• Connector-specific metadata in property definitions is not imported. LB3 model import: preserve connector-specific metadata in property definitions #3810
• When a property is defined with a complex object type, the nested property definitions are not converted from LB3 to LB4 format. LB3 model import: convert definition of nested properties #3811
• Model relations are not imported. Spike: how to import relations from LB3 models #3812
• Models inheriting from application-specific models (including LB3 built-in models like User) cannot be imported yet. Spike: how to import LB3 models extending a custom base model #3813
• For models attached to MongoDB datasource, the imported LB4 model contains incorrect definition of the primary key property of ObjectID type. LB3 model import: handle ObjectID type for MongoDB #3814

[dhmlau]

Closing as done!