Skip to content

Commit

Permalink
Merge pull request #229 from jacob-alford/json-schema
Browse files Browse the repository at this point in the history
[#137] Add JsonSchema Derivation
  • Loading branch information
jacob-alford authored Jan 5, 2023
2 parents 4723b3e + d39161d commit 0ab5162
Show file tree
Hide file tree
Showing 50 changed files with 1,972 additions and 74 deletions.
96 changes: 90 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,10 @@ yarn add schemata-ts
npm install schemata-ts
```

## Codec
## Codec and Arbitrary

A codec is a typeclass that contains the methods of `Decoder`, `Encoder`, `JsonSerializer`, `JsonDeserializer`, and `Guard`. Decoder and encoder are lossless when composed together. This means that for all domain types for which an encoder encodes to, a decoder will return a valid `E.Right` value.

Likewise, `JsonSerializer` and `JsonDeserializer` are lossless when composed together. Certain data types in Javascript like `NaN`, `undefined`, `Infinity`, and others are not part of the JSON specification, and `JSON.stringify` will turn these values into something different. This means that if you stringify these types and attempt to parse, you will get a different object than you originally started with. Additionally, JSON cannot stringify `bigint`, and cannot contain circular references. Under these circumstances `JSON.stringify` may throw an error.

`JsonSerializer` and `JsonDeserializer` are typeclasses included in a Codec which are lossless when composed together. So anything that successfully stringifies using `JsonSerializer` will successfully parse with `JsonDeserializer` and be equivalent objects. This is useful to avoid bugs when using JSON strings for storing data. Additionally, `JsonDeserializer` will decode the Json string into a domain type for immediate use in your program.

### User Document Example

This is a live example found in `src/Codec.ts` type-checked and tested with [docs-ts](https://github.com/gcanti/docs-ts).
Expand Down Expand Up @@ -140,6 +136,94 @@ assert.deepStrictEqual(
)
```

## Json Serializer and Deserializer

Like encoder and decoder, `JsonSerializer` and `JsonDeserializer` are lossless when composed together. Certain data types in Javascript like `NaN`, `undefined`, `Infinity`, and others are not part of the JSON specification, and `JSON.stringify` will turn these values into something different (or omit them). This means that if you stringify these types and attempt to parse, you will get a different object than you originally started with. Additionally, JSON cannot stringify `bigint`, and cannot contain circular references. Under these circumstances `JSON.stringify` will throw an error.

Anything that successfully stringifies using `JsonSerializer` will successfully parse with `JsonDeserializer` and will be equivalent objects. This is useful to avoid bugs when using JSON strings for storing data. Additionally, `JsonDeserializer` will decode the Json string into a domain type for immediate use in your program.

## Deriving JSON Schema

Schemata-ts comes with its own implementation of [JSON-Schema](https://json-schema.org/) and is a validation standard that can be used to validate artifacts in many other languages and frameworks. Schemata-ts's implementation is compatible with JSON Schema Draft 4, Draft 6, Draft 7, Draft 2019-09, and has partial support for 2020-12. _Note_: string `format` (like regex, contentType, or mediaType) is only available starting with Draft 6, and tuples are not compatible with Draft 2020-12.

### Customer JSON Schema Example

This is a live example generating a JSON Schema in `src/base/JsonSchemaBase.ts`

```ts
import * as JS from 'schemata-ts/base/JsonSchemaBase'
import * as S from 'schemata-ts/schemata'
import { getJsonSchema } from 'schemata-ts/JsonSchema'

const schema = S.Struct({
id: S.Natural,
jwt: S.Jwt,
tag: S.Literal('Customer'),
})

const jsonSchema = getJsonSchema(schema)

assert.deepStrictEqual(JS.stripIdentity(jsonSchema), {
type: 'object',
required: ['id', 'jwt', 'tag'],
properties: {
id: { type: 'integer', minimum: 0, maximum: 9007199254740991 },
jwt: {
type: 'string',
description: 'Jwt',
pattern:
'^(([A-Za-z0-9_\\x2d]*)\\.([A-Za-z0-9_\\x2d]*)(\\.([A-Za-z0-9_\\x2d]*)){0,1})$',
},
tag: { type: 'string', const: 'Customer' },
},
})
```

A note on `JS.stripIdentity` in the above example: internally, JSON Schema is represented as a union of Typescript classes. This is handy when inspecting the schema because the name of the schema is shown next to its properties. Because this prevents equality comparison, schemata-ts exposes a method `stripIdentity` to remove the object's class identity. _Caution_: this method stringifies and parses the schema and may throw if the schema itself contains circularity.

## Pattern Builder

Schemata-ts comes with powerful regex combinators that are used to construct regex from comprehensible atoms. The `Pattern` schema allows extension of a string schema to a subset of strings defined by a pattern. Decoders and Guards guarantee that a string conforms to the specified pattern, Arbitrary generates strings that follow the pattern, and Json Schema generates string schemata that lists the pattern as a property.

### US Phone Number Example

This is a live example validating US Phone numbers found in `src/PatternBuilder.ts`

```ts
import * as PB from 'schemata-ts/PatternBuilder'
import { pipe } from 'fp-ts/function'

const digit = PB.characterClass(false, ['0', '9'])

const areaCode = pipe(
pipe(
PB.char('('),
PB.then(PB.times(3)(digit)),
PB.then(PB.char(')')),
PB.then(PB.maybe(PB.char(' '))),
),
PB.or(PB.times(3)(digit)),
PB.subgroup,
)

const prefix = PB.times(3)(digit)

const lineNumber = PB.times(4)(digit)

export const usPhoneNumber = pipe(
areaCode,
PB.then(pipe(PB.char('-'), PB.maybe)),
PB.then(prefix),
PB.then(PB.char('-')),
PB.then(lineNumber),
)

assert.equal(PB.regexFromPattern(usPhoneNumber).test('(123) 456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('(123)456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('123-456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('1234567890'), false)
```

## Documentation

- [schemata-ts](https://jacob-alford.github.io/schemata-ts/docs/modules)
Expand Down Expand Up @@ -203,7 +287,7 @@ Additionally, there are more unlisted base schemable schemata also exported from
schemata-ts is planned to support the following features in various 1.x and 2.x versions in the near future:

- ~~JsonSerializer and JsonDeserializer: encoding / decoding from Json strings~~ Added in 1.1.0
- Json Schema: generating JSON-Schema from schemata ([#137](https://github.com/jacob-alford/schemata-ts/issues/137))
- ~~Json Schema: generating JSON-Schema from schemata ([#137](https://github.com/jacob-alford/schemata-ts/issues/137))~~ Added in 1.2.0
- Optic Schema: generating optics from schemata ([#134](https://github.com/jacob-alford/schemata-ts/issues/134))
- Mapped Structs: conversions between struct types, i.e. `snake-case` keys to `camelCase` keys
- More generic schemata: (SetFromArray, ~~NonEmptyArray~~ Added in 1.1.0)
Expand Down
110 changes: 106 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,27 @@ nav_order: 1

A schema is an expression of a type structure that can be used to generate typeclass instances from a single declaration. Typeclass instances can perform a variety of tasks, for instance `Decoder` can take a pesky `unknown` value and give you an Either in return where the success case abides by the `schema` that generated it. The example below constructs a `User` schema.

## Codec
## Installation

A codec is a typeclass that contains the methods of `Decoder`, `Encoder`, `JsonSerializer`, `JsonDeserializer`, and `Guard`. Decoder and encoder are lossless when composed together. This means that for all domain types for which an encoder encodes to, a decoder will return a valid `E.Right` value.
Uses `fp-ts`, and `io-ts` as peer dependencies. Read more about peer dependencies at [nodejs.org](https://nodejs.org/en/blog/npm/peer-dependencies/).

Also contains `fast-check` as a soft peer dependency. Soft peer dependency implies that usage of the `Arbitrary` module requires fast-check as a peer-dependency.

### Yarn

```bash
yarn add schemata-ts
```

Likewise, `JsonSerializer` and `JsonDeserializer` are lossless when composed together. Certain data types in Javascript like `NaN`, `undefined`, `Infinity`, and others are not part of the JSON specification, and `JSON.stringify` will turn these values into something different. This means that if you stringify these types and attempt to parse, you will get a different object than you originally started with. Additionally, JSON cannot stringify `bigint`, and cannot contain circular references. Under these circumstances `JSON.stringify` may throw an error.
### NPM

`JsonSerializer` and `JsonDeserializer` are typeclasses included in a Codec which are lossless when composed together. So anything that successfully stringifies using `JsonSerializer` will successfully parse with `JsonDeserializer` and be equivalent objects. This is useful to avoid bugs when using JSON strings for storing data. Additionally, `JsonDeserializer` will decode the Json string into a domain type for immediate use in your program.
```bash
npm install schemata-ts
```

## Codec and Arbitrary

A codec is a typeclass that contains the methods of `Decoder`, `Encoder`, `JsonSerializer`, `JsonDeserializer`, and `Guard`. Decoder and encoder are lossless when composed together. This means that for all domain types for which an encoder encodes to, a decoder will return a valid `E.Right` value.

### User Document Example

Expand Down Expand Up @@ -114,3 +128,91 @@ assert.deepStrictEqual(
E.right(testUsers),
)
```

## Json Serializer and Deserializer

Like encoder and decoder, `JsonSerializer` and `JsonDeserializer` are lossless when composed together. Certain data types in Javascript like `NaN`, `undefined`, `Infinity`, and others are not part of the JSON specification, and `JSON.stringify` will turn these values into something different (or omit them). This means that if you stringify these types and attempt to parse, you will get a different object than you originally started with. Additionally, JSON cannot stringify `bigint`, and cannot contain circular references. Under these circumstances `JSON.stringify` will throw an error.

Anything that successfully stringifies using `JsonSerializer` will successfully parse with `JsonDeserializer` and will be equivalent objects. This is useful to avoid bugs when using JSON strings for storing data. Additionally, `JsonDeserializer` will decode the Json string into a domain type for immediate use in your program.

## Deriving JSON Schema

Schemata-ts comes with its own implementation of [JSON-Schema](https://json-schema.org/) and is a validation standard that can be used to validate artifacts in many other languages and frameworks. Schemata-ts's implementation is compatible with JSON Schema Draft 4, Draft 6, Draft 7, Draft 2019-09, and has partial support for 2020-12. _Note_: string `format` (like regex, contentType, or mediaType) is only available starting with Draft 6, and tuples are not compatible with Draft 2020-12.

### Customer JSON Schema Example

This is a live example generating a JSON Schema in `src/base/JsonSchemaBase.ts`

```ts
import * as JS from 'schemata-ts/base/JsonSchemaBase'
import * as S from 'schemata-ts/schemata'
import { getJsonSchema } from 'schemata-ts/JsonSchema'

const schema = S.Struct({
id: S.Natural,
jwt: S.Jwt,
tag: S.Literal('Customer'),
})

const jsonSchema = getJsonSchema(schema)

assert.deepStrictEqual(JS.stripIdentity(jsonSchema), {
type: 'object',
required: ['id', 'jwt', 'tag'],
properties: {
id: { type: 'integer', minimum: 0, maximum: 9007199254740991 },
jwt: {
type: 'string',
description: 'Jwt',
pattern:
'^(([A-Za-z0-9_\\x2d]*)\\.([A-Za-z0-9_\\x2d]*)(\\.([A-Za-z0-9_\\x2d]*)){0,1})$',
},
tag: { type: 'string', const: 'Customer' },
},
})
```

A note on `JS.stripIdentity` in the above example: internally, JSON Schema is represented as a union of Typescript classes. This is handy when inspecting the schema because the name of the schema is shown next to its properties. Because this prevents equality comparison, schemata-ts exposes a method `stripIdentity` to remove the object's class identity. _Caution_: this method stringifies and parses the schema and may throw if the schema itself contains circularity.

## Pattern Builder

Schemata-ts comes with powerful regex combinators that are used to construct regex from comprehensible atoms. The `Pattern` schema allows extension of a string schema to a subset of strings defined by a pattern. Decoders and Guards guarantee that a string conforms to the specified pattern, Arbitrary generates strings that follow the pattern, and Json Schema generates string schemata that lists the pattern as a property.

### US Phone Number Example

This is a live example validating US Phone numbers found in `src/PatternBuilder.ts`

```ts
import * as PB from 'schemata-ts/PatternBuilder'
import { pipe } from 'fp-ts/function'

const digit = PB.characterClass(false, ['0', '9'])

const areaCode = pipe(
pipe(
PB.char('('),
PB.then(PB.times(3)(digit)),
PB.then(PB.char(')')),
PB.then(PB.maybe(PB.char(' '))),
),
PB.or(PB.times(3)(digit)),
PB.subgroup,
)

const prefix = PB.times(3)(digit)

const lineNumber = PB.times(4)(digit)

export const usPhoneNumber = pipe(
areaCode,
PB.then(pipe(PB.char('-'), PB.maybe)),
PB.then(prefix),
PB.then(PB.char('-')),
PB.then(lineNumber),
)

assert.equal(PB.regexFromPattern(usPhoneNumber).test('(123) 456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('(123)456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('123-456-7890'), true)
assert.equal(PB.regexFromPattern(usPhoneNumber).test('1234567890'), false)
```
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "schemata-ts",
"version": "1.1.0",
"version": "1.2.0",
"description": "A collection of Schemata inspired by io-ts-types and validators.js",
"homepage": "https://jacob-alford.github.io/schemata-ts/",
"repository": {
Expand Down Expand Up @@ -94,6 +94,7 @@
"glob": "^8.0.3",
"husky": "^8.0.0",
"jest": "^26.6.3",
"json-schema-library": "^7.4.4",
"lint-staged": "^13.0.3",
"markdown-magic": "^2.0.0",
"prettier": "^2.2.1",
Expand Down
2 changes: 2 additions & 0 deletions scripts/generate-schemables.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ export type SchemableTypeclasses =
| SchemableTypeclass<'Encoder', 'Enc', 'SchemableExt2', '1.0.0'>
| SchemableTypeclass<'Arbitrary', 'Arb', 'SchemableExt1', '1.0.0'>
| SchemableTypeclass<'Printer', 'P', 'SchemableExt2', '1.1.0'>
| SchemableTypeclass<'JsonSchema', 'JS', 'SchemableExt2', '1.2.0'>

// #region Typeclass modules

Expand Down Expand Up @@ -359,6 +360,7 @@ const schemableTypeclasses: ReadonlyArray<SchemableTypeclasses> = [
['Encoder', 'Enc', 'SchemableExt2', '1.0.0', 'encoder'],
['Arbitrary', 'Arb', 'SchemableExt1', '1.0.0', 'arbitrary'],
['Printer', 'P', 'SchemableExt2', '1.1.0', 'printer'],
['JsonSchema', 'JS', 'SchemableExt2', '1.2.0', 'json-schema'],
]

const format: Build<void> = C => C.exec('yarn format')
Expand Down
2 changes: 2 additions & 0 deletions src/Arbitrary.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
import * as Arb from './base/ArbitraryBase'
import { SchemableExt1 } from './SchemableExt'
import * as WithAnnotate from './schemables/WithAnnotate/instances/arbitrary'
import * as WithBrand from './schemables/WithBrand/instances/arbitrary'
import * as WithCheckDigit from './schemables/WithCheckDigit/instances/arbitrary'
import * as WithDate from './schemables/WithDate/instances/arbitrary'
Expand Down Expand Up @@ -36,6 +37,7 @@ export type {
*/
export const Schemable: SchemableExt1<Arb.URI> = {
...Arb.Schemable,
...WithAnnotate.Arbitrary,
...WithBrand.Arbitrary,
...WithCheckDigit.Arbitrary,
...WithDate.Arbitrary,
Expand Down
2 changes: 2 additions & 0 deletions src/Decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
import * as D from './base/DecoderBase'
import { SchemableExt2C } from './SchemableExt'
import * as WithAnnotate from './schemables/WithAnnotate/instances/decoder'
import * as WithBrand from './schemables/WithBrand/instances/decoder'
import * as WithCheckDigit from './schemables/WithCheckDigit/instances/decoder'
import * as WithDate from './schemables/WithDate/instances/decoder'
Expand Down Expand Up @@ -36,6 +37,7 @@ export type {
*/
export const Schemable: SchemableExt2C<D.URI> = {
...D.Schemable,
...WithAnnotate.Decoder,
...WithBrand.Decoder,
...WithCheckDigit.Decoder,
...WithDate.Decoder,
Expand Down
2 changes: 2 additions & 0 deletions src/Encoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
import * as Enc from './base/EncoderBase'
import { SchemableExt2 } from './SchemableExt'
import * as WithAnnotate from './schemables/WithAnnotate/instances/encoder'
import * as WithBrand from './schemables/WithBrand/instances/encoder'
import * as WithCheckDigit from './schemables/WithCheckDigit/instances/encoder'
import * as WithDate from './schemables/WithDate/instances/encoder'
Expand Down Expand Up @@ -36,6 +37,7 @@ export type {
*/
export const Schemable: SchemableExt2<Enc.URI> = {
...Enc.Schemable,
...WithAnnotate.Encoder,
...WithBrand.Encoder,
...WithCheckDigit.Encoder,
...WithDate.Encoder,
Expand Down
2 changes: 2 additions & 0 deletions src/Eq.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
import * as Eq from './base/EqBase'
import { SchemableExt1 } from './SchemableExt'
import * as WithAnnotate from './schemables/WithAnnotate/instances/eq'
import * as WithBrand from './schemables/WithBrand/instances/eq'
import * as WithCheckDigit from './schemables/WithCheckDigit/instances/eq'
import * as WithDate from './schemables/WithDate/instances/eq'
Expand Down Expand Up @@ -36,6 +37,7 @@ export type {
*/
export const Schemable: SchemableExt1<Eq.URI> = {
...Eq.Schemable,
...WithAnnotate.Eq,
...WithBrand.Eq,
...WithCheckDigit.Eq,
...WithDate.Eq,
Expand Down
2 changes: 2 additions & 0 deletions src/Guard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
import * as G from './base/GuardBase'
import { SchemableExt1 } from './SchemableExt'
import * as WithAnnotate from './schemables/WithAnnotate/instances/guard'
import * as WithBrand from './schemables/WithBrand/instances/guard'
import * as WithCheckDigit from './schemables/WithCheckDigit/instances/guard'
import * as WithDate from './schemables/WithDate/instances/guard'
Expand Down Expand Up @@ -36,6 +37,7 @@ export type {
*/
export const Schemable: SchemableExt1<G.URI> = {
...G.Schemable,
...WithAnnotate.Guard,
...WithBrand.Guard,
...WithCheckDigit.Guard,
...WithDate.Guard,
Expand Down
Loading

0 comments on commit 0ab5162

Please sign in to comment.