Skip to content

Commit

Permalink
Refactor Bindgen to simpler form
Browse files Browse the repository at this point in the history
Refactoring the bindgen with the goal of having a simpler IR-like form
that makes recursively generating types and code easier.

The end goal is to remove all the hairy type portions of the XTP Schema
document. e.g.:

* type
* format
* items
* $ref
* additionalProperties

and replace with a recursively defined type `XtpNormalizedSchema` (name
will likely change to XtpType). I've kept things backwards compatible by
only adding this type on the property `xtpType`. We should switch
bindgens to use this object then we can remove the other properties.

I also decided to change a little bit about the whole process. e.g.:

* moved validation to the parser level code
* remove circularReference detection and blocking
* added some more tests

In order to think more clearly about the whole process, I took up a moment to
write up what I think the whole flow for schemas should be in terms of validating and compiling:

First we validate against JSON schema. This should ensure that we can parse the document into typescript types in the next step. It should not allow any extra values or any values outside of the enum ranges. We should be able to do a simple `const doc = rawDoc as V1Schema` in typescript and the doc object should be valid.

Here we parse the json or yaml into a raw javascript object and cast it to the V1 or V0 schema type. This gives us a raw, but typed representation of the schema. Here we should do extra conditional validation steps that can’t be done (or are too complex to be done) in the JSON Schema. e.g. validating content-type / type pairs, etc.

Here we take the raw parsed types and “normalize” them into a simpler form. We will walk the document and replace all occurrences of $ref, items, additionalProperties, type, and format with a single recursive XtpType.
  • Loading branch information
bhelx committed Oct 28, 2024
1 parent 2839ebb commit cfc35d7
Show file tree
Hide file tree
Showing 12 changed files with 837 additions and 487 deletions.
77 changes: 52 additions & 25 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,15 @@
"devDependencies": {
"@extism/js-pdk": "^1.0.1",
"@types/jest": "^29.5.12",
"@types/js-yaml": "^4.0.9",
"@types/node": "^22.8.1",
"const": "^1.0.0",
"esbuild": "^0.17.0",
"esbuild-plugin-d.ts": "^1.2.3",
"jest": "^29.0.0",
"js-yaml": "^4.1.0",
"ts-jest": "^29.0.0",
"typescript": "^5.0.0"
"typescript": "^5.6.3"
},
"files": [
"dist"
Expand Down
10 changes: 5 additions & 5 deletions src/common.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export class ValidationError extends Error {
constructor(public message: string, public path: string) {
super(message);
Object.setPrototypeOf(this, ValidationError.prototype);
}
export class ValidationError {
constructor(public message: string, public path: string) {
this.message = message
this.path = path
}
}
53 changes: 50 additions & 3 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import {
XtpSchema,
} from "./normalizer";
import { CodeSample } from "./parser";
import { XtpNormalizedType } from "./types";
export * from "./normalizer";
export * from "./types";
export { ValidationError } from "./common";

export function parse(schema: string) {
Expand Down Expand Up @@ -104,9 +106,43 @@ function isPrimitive(p: Property | Parameter): boolean {
return !!p.$ref.enum || !p.$ref.properties;
}

function isDateTime(p: Property | Parameter | null): boolean {
if (!p) return false;
return p.type === "string" && p.format === "date-time";
export type XtpTyped = { xtpType: XtpNormalizedType } | null;

function isDateTime(p: XtpTyped): boolean {
return p?.xtpType?.kind === 'date-time'
}
function isBuffer(p: XtpTyped): boolean {
return p?.xtpType?.kind === "buffer"
}
function isObject(p: XtpTyped): boolean {
return p?.xtpType?.kind === "object"
}
function isArray(p: XtpTyped): boolean {
return p?.xtpType?.kind === "array"
}
function isEnum(p: XtpTyped): boolean {
return p?.xtpType?.kind === "enum"
}
function isString(p: XtpTyped): boolean {
return p?.xtpType?.kind === "string"
}
function isInt32(p: XtpTyped): boolean {
return p?.xtpType?.kind === "int32"
}
function isInt64(p: XtpTyped): boolean {
return p?.xtpType?.kind === "int64"
}
function isFloat(p: XtpTyped): boolean {
return p?.xtpType?.kind === "float"
}
function isDouble(p: XtpTyped): boolean {
return p?.xtpType?.kind === "double"
}
function isBoolean(p: XtpTyped): boolean {
return p?.xtpType?.kind === "boolean"
}
function isMap(p: XtpTyped): boolean {
return p?.xtpType?.kind === "map"
}

function capitalize(s: string) {
Expand Down Expand Up @@ -135,6 +171,17 @@ export const helpers = {
codeSamples,
isDateTime,
isPrimitive,
isBuffer,
isObject,
isEnum,
isArray,
isString,
isInt32,
isInt64,
isFloat,
isDouble,
isMap,
isBoolean,
isJsonEncoded,
isUtf8Encoded,
capitalize,
Expand Down
Loading

0 comments on commit cfc35d7

Please sign in to comment.