diff --git a/docs/cli/commands-description.md b/docs/cli/commands-description.md index dd7d7bd28..18660da70 100644 --- a/docs/cli/commands-description.md +++ b/docs/cli/commands-description.md @@ -416,63 +416,15 @@ Remove existing profile configuration --- -## $ si scope | s +## $ si scope **Description** -/This functionality is under development./ Manage scopes that store pairs of spaces and Hubs used when working -**Usage** - -`si scope [command] [options...]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si scope list | ls - -**Description** - -List all created scopes - -**Usage** - -`si scope list [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si scope print - -**Description** - -See json file under the scope - -**Usage** - -`si scope print [options] ` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si scope use - -**Description** - -Work on the selected scope **Usage** -`si scope use [options] ` +`si scope [options]` **Options** @@ -480,49 +432,15 @@ Work on the selected scope --- -## $ si scope delete +## $ si space **Description** -Delete specific scope - -**Usage** - -`si scope delete [options] ` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space | spc - -**Description** - -Operations on grouped and separated runtime environments that allow sharing the data within them - -**Usage** - -`si space [command] [options...]` -**Options** - -* -c, --stdout Output to stdout (ignores -o) -* -o, --output Output path - defaults to dirname -* -h, --help Display help for command - ---- - -## $ si space info - -**Description** - -Display info about the default space **Usage** -`si space info [options]` +`si space [options]` **Options** @@ -530,160 +448,6 @@ Display info about the default space --- -## $ si space list | ls - -**Description** - -List all existing spaces - -**Usage** - -`si space list [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space use - -**Description** - -Use the space - -**Usage** - -`si space use [options] ` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space audit - -**Description** - -Fetch all audit messages from spaces - -**Usage** - -`si space audit [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space logs - -**Description** - -Fetch all logs from space - -**Usage** - -`si space logs [options] []` - -**Arguments** - -* The name of the space (defaults to current space) - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space version - -**Description** - -Display space version - -**Usage** - -`si space version [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space access - -**Description** - -Manages Access Keys for active Space - -**Usage** - -`si space access [options] [command]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space access create - -**Description** - -Create Access key for adding Hubs to active Space, i.e "Army of Darkness" - -**Usage** - -`si space access create [options] ` - -**Arguments** - -* description Key description - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space access list | ls - -**Description** - -List Access Keys metadata in active Space - -**Usage** - -`si space access list [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si space access revoke - -**Description** - -Revokes Access Key in active Space - -**Usage** - -`si space access revoke [options]` - -**Options** - -* --id revoke specified key -* --all Removes all access keys and disconnects all self-hosted Hubs connected to Space -* -h, --help Display help for command - ---- - ## $ si hub **Description** @@ -696,97 +460,6 @@ Allows to run programs in different data centers, computers or devices in local **Options** -* -h, --help Display help for command - ---- - -## $ si hub use - -**Description** - -Specify the default Hub you want to work with, all subsequent requests will be sent to this Hub - -**Usage** - -`si hub use [options] ` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si hub list | ls - -**Description** - -List all the Hubs in the default space - -**Usage** - -`si hub list [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si hub info - -**Description** - -Display info about the default Hub - -**Usage** - -`si hub info [options]` - -**Options** - -* -h, --help Display help for command - ---- - -## $ si hub disconnect - -**Description** - -Disconnect self hosted Hubs from space - -**Usage** - -`si hub disconnect [options] ` - -**Arguments** - -* space_name The name of the Space - -**Options** - -* --id Hub Id -* --all Disconnects all self-hosted Hubs connected to Space (default: false) -* -h, --help Display help for command - ---- - -## $ si hub delete - -**Description** - -Delete self hosted Hub from space - -**Usage** - -`si hub delete [options] ` - -**Arguments** - -* id Hub Id - -**Options** - -* -f, --force Enable deleting Hubs that are not disconnected (default: false) * -h, --help Display help for command --- @@ -1482,8 +1155,8 @@ Create topic **Options** -* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", "application/octet-stream", default: -* "application/x-ndjson") +* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", +* "application/octet-stream", default: "application/x-ndjson") * -h, --help Display help for command --- @@ -1516,8 +1189,8 @@ Get data from topic **Options** -* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", "application/octet-stream", default: -* "application/x-ndjson") +* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", +* "application/octet-stream", default: "application/x-ndjson") * -h, --help Display help for command --- @@ -1534,8 +1207,8 @@ Send data on topic from file, directory or directly through the console **Options** -* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", "application/octet-stream", default: -* "application/x-ndjson") +* -t, --content-type [content-type] Specifies type of data in topic (choices: "text/x-ndjson", "application/x-ndjson", "text/plain", +* "application/octet-stream", default: "application/x-ndjson") * -h, --help Display help for command --- @@ -1556,6 +1229,22 @@ List information about topics --- +## $ si store + +**Description** + + + +**Usage** + +`si store [options]` + +**Options** + +* -h, --help Display help for command + +--- + ## $ si util | u **Description** diff --git a/docs/host/classes/Host.md b/docs/host/classes/Host.md index 4bee9fb45..43ad39695 100644 --- a/docs/host/classes/Host.md +++ b/docs/host/classes/Host.md @@ -87,7 +87,7 @@ Can communicate with Manager. #### Defined in -[packages/host/src/lib/host.ts:144](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L144) +[packages/host/src/lib/host.ts:147](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L147) ___ @@ -99,7 +99,7 @@ The Host's API Server. #### Defined in -[packages/host/src/lib/host.ts:92](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L92) +[packages/host/src/lib/host.ts:95](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L95) ___ @@ -111,7 +111,7 @@ Api path prefix based on initial configuration. #### Defined in -[packages/host/src/lib/host.ts:97](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L97) +[packages/host/src/lib/host.ts:100](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L100) ___ @@ -123,7 +123,7 @@ Host auditor. #### Defined in -[packages/host/src/lib/host.ts:78](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L78) +[packages/host/src/lib/host.ts:81](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L81) ___ @@ -133,7 +133,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:138](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L138) +[packages/host/src/lib/host.ts:141](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L141) ___ @@ -145,7 +145,7 @@ Configuration. #### Defined in -[packages/host/src/lib/host.ts:87](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L87) +[packages/host/src/lib/host.ts:90](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L90) ___ @@ -157,7 +157,7 @@ Instance of CPMConnector used to communicate with Manager. #### Defined in -[packages/host/src/lib/host.ts:111](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L111) +[packages/host/src/lib/host.ts:114](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L114) ___ @@ -167,7 +167,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:142](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L142) +[packages/host/src/lib/host.ts:145](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L145) ___ @@ -179,7 +179,7 @@ Instance path prefix. #### Defined in -[packages/host/src/lib/host.ts:102](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L102) +[packages/host/src/lib/host.ts:105](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L105) ___ @@ -195,7 +195,7 @@ Object to store CSIControllers. #### Defined in -[packages/host/src/lib/host.ts:116](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L116) +[packages/host/src/lib/host.ts:119](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L119) ___ @@ -205,7 +205,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:143](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L143) +[packages/host/src/lib/host.ts:146](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L146) ___ @@ -217,7 +217,7 @@ Instance of class providing load check. #### Defined in -[packages/host/src/lib/host.ts:131](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L131) +[packages/host/src/lib/host.ts:134](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L134) ___ @@ -233,7 +233,7 @@ IComponent.logger #### Defined in -[packages/host/src/lib/host.ts:126](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L126) +[packages/host/src/lib/host.ts:129](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L129) ___ @@ -243,7 +243,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:140](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L140) +[packages/host/src/lib/host.ts:143](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L143) ___ @@ -255,7 +255,7 @@ S3 client. #### Defined in -[packages/host/src/lib/host.ts:149](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L149) +[packages/host/src/lib/host.ts:152](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L152) ___ @@ -267,7 +267,7 @@ Sequences store. #### Defined in -[packages/host/src/lib/host.ts:121](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L121) +[packages/host/src/lib/host.ts:124](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L124) ___ @@ -279,7 +279,7 @@ Service to handle topics. #### Defined in -[packages/host/src/lib/host.ts:136](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L136) +[packages/host/src/lib/host.ts:139](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L139) ___ @@ -289,7 +289,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:106](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L106) +[packages/host/src/lib/host.ts:109](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L109) ___ @@ -299,7 +299,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:80](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L80) +[packages/host/src/lib/host.ts:83](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L83) ___ @@ -309,7 +309,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:82](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L82) +[packages/host/src/lib/host.ts:85](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L85) ___ @@ -319,7 +319,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:104](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L104) +[packages/host/src/lib/host.ts:107](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L107) ## Accessors @@ -333,7 +333,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:172](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L172) +[packages/host/src/lib/host.ts:175](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L175) ___ @@ -347,7 +347,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:182](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L182) +[packages/host/src/lib/host.ts:185](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L185) ___ @@ -361,7 +361,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:168](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L168) +[packages/host/src/lib/host.ts:171](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L171) ___ @@ -375,7 +375,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:178](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L178) +[packages/host/src/lib/host.ts:181](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L181) ## Methods @@ -397,7 +397,7 @@ Setting up handlers for general Host API endpoints: #### Defined in -[packages/host/src/lib/host.ts:470](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L470) +[packages/host/src/lib/host.ts:497](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L497) ___ @@ -413,7 +413,7 @@ Stops running servers. #### Defined in -[packages/host/src/lib/host.ts:1188](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1188) +[packages/host/src/lib/host.ts:1215](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1215) ___ @@ -429,7 +429,7 @@ Initializes connector and connects to Manager. #### Defined in -[packages/host/src/lib/host.ts:442](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L442) +[packages/host/src/lib/host.ts:469](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L469) ___ @@ -449,7 +449,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:850](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L850) +[packages/host/src/lib/host.ts:877](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L877) ___ @@ -463,7 +463,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:253](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L253) +[packages/host/src/lib/host.ts:280](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L280) ___ @@ -481,7 +481,7 @@ List of Instances. #### Defined in -[packages/host/src/lib/host.ts:1091](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1091) +[packages/host/src/lib/host.ts:1118](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1118) ___ @@ -505,7 +505,7 @@ Sequence info object. #### Defined in -[packages/host/src/lib/host.ts:1103](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1103) +[packages/host/src/lib/host.ts:1130](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1130) ___ @@ -529,7 +529,7 @@ List of Instances. #### Defined in -[packages/host/src/lib/host.ts:1140](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1140) +[packages/host/src/lib/host.ts:1167](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1167) ___ @@ -547,7 +547,7 @@ List of Sequences. #### Defined in -[packages/host/src/lib/host.ts:1128](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1128) +[packages/host/src/lib/host.ts:1155](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1155) ___ @@ -565,7 +565,7 @@ Size #### Defined in -[packages/host/src/lib/host.ts:1254](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1254) +[packages/host/src/lib/host.ts:1281](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1281) ___ @@ -579,7 +579,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:1159](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1159) +[packages/host/src/lib/host.ts:1186](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1186) ___ @@ -593,7 +593,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:1153](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1153) +[packages/host/src/lib/host.ts:1180](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1180) ___ @@ -614,7 +614,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:688](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L688) +[packages/host/src/lib/host.ts:715](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L715) ___ @@ -641,7 +641,7 @@ Promise resolving to operation result object. #### Defined in -[packages/host/src/lib/host.ts:601](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L601) +[packages/host/src/lib/host.ts:628](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L628) ___ @@ -662,7 +662,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:732](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L732) +[packages/host/src/lib/host.ts:759](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L759) ___ @@ -689,7 +689,7 @@ Promise resolving to operation result. #### Defined in -[packages/host/src/lib/host.ts:830](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L830) +[packages/host/src/lib/host.ts:857](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L857) ___ @@ -709,7 +709,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:802](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L802) +[packages/host/src/lib/host.ts:829](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L829) ___ @@ -737,7 +737,7 @@ Promise resolving to operation result object. #### Defined in -[packages/host/src/lib/host.ts:891](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L891) +[packages/host/src/lib/host.ts:918](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L918) ___ @@ -751,7 +751,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:667](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L667) +[packages/host/src/lib/host.ts:694](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L694) ___ @@ -768,7 +768,7 @@ Used to recover Sequences information after restart. #### Defined in -[packages/host/src/lib/host.ts:706](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L706) +[packages/host/src/lib/host.ts:733](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L733) ___ @@ -798,7 +798,7 @@ Instance middleware. #### Defined in -[packages/host/src/lib/host.ts:538](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L538) +[packages/host/src/lib/host.ts:565](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L565) ___ @@ -818,7 +818,7 @@ Promise resolving to Instance of Host. #### Defined in -[packages/host/src/lib/host.ts:300](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L300) +[packages/host/src/lib/host.ts:327](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L327) ___ @@ -832,7 +832,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:389](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L389) +[packages/host/src/lib/host.ts:416](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L416) ___ @@ -854,7 +854,7 @@ ___ #### Defined in -[packages/host/src/lib/host.ts:1261](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1261) +[packages/host/src/lib/host.ts:1288](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1288) ___ @@ -872,7 +872,7 @@ Configuration object. #### Defined in -[packages/host/src/lib/host.ts:271](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L271) +[packages/host/src/lib/host.ts:298](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L298) ___ @@ -888,7 +888,7 @@ Sets up telemetry. #### Defined in -[packages/host/src/lib/host.ts:1228](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1228) +[packages/host/src/lib/host.ts:1255](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1255) ___ @@ -912,7 +912,7 @@ Forward request to Manager the Host is connected to. #### Defined in -[packages/host/src/lib/host.ts:570](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L570) +[packages/host/src/lib/host.ts:597](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L597) ___ @@ -935,7 +935,7 @@ Creates new CSIController [CSIController](CSIController.md) object and handles i #### Defined in -[packages/host/src/lib/host.ts:977](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L977) +[packages/host/src/lib/host.ts:1004](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1004) ___ @@ -952,7 +952,7 @@ using its CSIController [CSIController](CSIController.md) #### Defined in -[packages/host/src/lib/host.ts:1171](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1171) +[packages/host/src/lib/host.ts:1198](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L1198) ## Constructors @@ -973,4 +973,4 @@ Sets used modules with provided configuration. #### Defined in -[packages/host/src/lib/host.ts:194](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L194) +[packages/host/src/lib/host.ts:197](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/host/src/lib/host.ts#L197) diff --git a/docs/module-loader/README.md b/docs/module-loader/README.md new file mode 100644 index 000000000..ef60f3376 --- /dev/null +++ b/docs/module-loader/README.md @@ -0,0 +1,743 @@ +@scramjet/module-loader / [Exports](modules.md) + +

Scramjet Transform Hub

+

+ GitHub license + STH version + GitHub stars + npm + Discord + + Donate + +

+

⭐ Star us on GitHub β€” it motivates us a lot! πŸš€

+

+ Scramjet Transform Hub Logo +

+ +# The Idea + +Scramjet Transform Hub is a cool way to run programs on your computer or in the cloud. It's easy to use, and it supports lots of different types of programs. + +The Hub is the heart of Scramjet Cloud Platform, a serverless data processing system that allows you to easily deploy, run and interconnect programs that process repetitive data tasks in long-running processes. STH can be run just as well on a Raspberry Pi as it can on a massive 128 core Epyc bare metal server. It installs in one simple command and deploys your app to processes, Docker containers or Kubernetes clusters just as easily... Oh, and the apps are incredibly simple to write too, here's a slack posting sample: + +```js +export (input: Readable) => async function*() { + for (const post of input) { + yield await slack.note(`New post titled "${post.title}" from ${post.author}`); + } +} +``` + +It currently supports Node.js and Python based apps, but Java and other languages are coming. + +[Get Scramjet Transform Hub straight from NPM](https://www.npmjs.com/package/@scramjet/sth) + +If you don't have a server ready or want to run something from the cloud itself, do check out our website and try the hosted version of [Scramjet Cloud Platform](https://scramjet.org/). + +--- + +# Table of contents + +- [Introduction :handshake:](#introduction-handshake) +- [Usage :bulb:](#usage-bulb) +- [The basics :abcd:](#the-basics-abcd) +- [Development instructions :construction_worker:](#development-instructions-construction_worker) + - [Installation :clamp:](#installation-clamp) + - [Environment setup](#environment-setup) + - [STH installation](#sth-installation) + - [Start STH :checkered_flag:](#start-sth-checkered_flag) + - [Install CLI and execute :white_check_mark:](#install-cli-and-execute-white_check_mark) + - [Build the packages :building_construction:](#build-the-packages-building_construction) + - [Docker commands :whale:](#docker-commands-whale) + - [Build Hub on Docker :building_construction:](#build-hub-on-docker-building_construction) + - [Run Transform Hub in Docker :robot:](#run-transform-hub-in-docker-robot) + - [Monorepo commands :pencil:](#monorepo-commands-pencil) + - [Autocomplete :zap:](#autocomplete-zap) + - [Git branching workflow](#git-branching-workflow) +- [Sample usage :sunglasses:](#sample-usage-sunglasses) + - ["Hello Alice" sample :wave:](#hello-alice-sample-wave) + - [More samples :books:](#more-samples-books) + - [Configure your own sample :pencil:](#configure-your-own-sample-pencil) +- [Troubleshooting :collision:](#troubleshooting-collision) +- [Start your 30 day trial :partying_face:](#start-your-30-days-trial-now-partying_face) +- [License and contributions :page_with_curl:](#license-and-contributions-page_with_curl) +- [Help wanted :information_desk_person:](#help-wanted-information_desk_person) +- [Donation :money_with_wings:](#donation-money_with_wings) + +--- + +# Introduction :handshake: + +This is a development repo for Scramjet Transform Hub, a container supervisor that allows deployment, execution and monitoring of any application based on a simple interface. + +This repository contains the source code for [Scramjet Transform Hub](https://docs.scramjet.org/platform/transform-hub). + +If you want to read more about the usage please go to [Quick Start](https://github.com/scramjetorg/platform-docs) repo. + +You may also be interested in our [Scramjet Cloud Platform](https://docs.scramjet.org/platform) that is based on Transform-Hub. + +What's more, this repository is intended for developers who would like to: + +- participate in the Scramjet Transform Hub development community +- register feature requests, issues and PRs for this product +- build Transform Hub from source +- dive into code and see how it works + +For developers who would like to simply use Scramjet Transform Hub to run data processing applications, we recommend following resources: + +![gh_logo](./images/gh_logo.png) [Repository with Introductory README and templates](https://github.com/scramjetorg/platform-docs) + +![gh_logo](./images/gh_logo.png) [Repository with samples](https://github.com/scramjetorg/platform-samples) + +![npm_logo](./images/npm_logo.png) [NPM page for main scramjet/sth package](https://www.npmjs.com/package/@scramjet/sth) + +![npm_logo](./images/npm_logo.png) [NPM page for scramjet/cli package](https://www.npmjs.com/package/@scramjet/cli) + +--- + +# Usage :bulb: + +The easiest way to start Transform Hub is to run the follow (you'll need node.js and npm installed): + +```bash +npx @scramjet/sth --runtime-adapter= +``` + +Scramjet Transform Hub is a platform that helps you run programs on local and remote machines. You can start programs in the background or connect to them and see their output. You can also pipe your data to the program even if it is running on a different machine. You can install this platform in different places like AWS, Google Cloud, Azure or even your own computer. + +There's no limit to what you can use it for. Do you want a stock checker? A chatbot? Maybe you'd like to automate your home? Retrieve sensor data? Maybe you have a lot of data and want to transfer and wrangle it? You have a database of cities and you'd like to enrich your data? You do machine learning and you want to train your set while the data is fetched in real-time? Or perhaps you want to use it for something else and ask us if that's a good use? Ask us [via email](mailto:get@scramjet.org) πŸ“§ or hop on our [Scramjet Discord ![discord](./images/discord.png)](https://scr.je/join-community-mg1) and we'll be happy to talk with you. + +--- + +# The basics :abcd: + +**NOTE:** This is the STH development repo and in order to use it, you need to have **Unix/Linux** based operating system, for example [Ubuntu](https://ubuntu.com/download/server) or [Windows with WSL2](https://learn.microsoft.com/en-us/windows/wsl/install). We also use [docker](https://www.docker.com/get-started) and [node.js v16.x](https://nodejs.org/en/). We're working on development guides for Mac and Windows. πŸ”œ + +Scramjet Transform Hub allows you to deploy and execute programs that you build and develop. As mentioned above, you can run any program you like, but you need to know a couple of important things: + +- The program should consist of a function or an array of functions, such a program is called a **Transform Sequence**. +- The Sequence will be executed within a separate docker instance (πŸ”œ we're working on other execution environment integrations - help will be appreciated 🦾). +- The Sequence function will receive a stream as input in the first argument - you can send the data to it via the command `si instance input`. +- If your Sequence contains more than one function, then the output from the first function is passed to the next one. Also, the first function in Sequence receives the input from API. +- The functions can also be Generators and iterate over input data in `node` and `python` versions. This is the simplest way to process data, but it won't leverage asynchronous processing. Check out our frameworks to run your transforms faster [Scramjet Framework](https://github.com/scramjetorg/scramjet#scramjet-framework) +- The last (or the only) function in Sequence can return a `Promise` or a `Stream` - based on this, STH will know when processing is done. +- Once the returned `Promise` or `future` is resolved, or the `Stream` is ended, STH will gracefully stop the Sequence and remove its container. +- You can communicate with the server via API, command line client `si` which we wrote for your convenience. +- The Sequence is called with an AppContext as `this` or `self`, a class that allows you to communicate back from the Sequence: send logs, provide health info, send and receive events from the API or CLI. +- You can run your Sequence multiple times with different arguments (like for instance currency tickers with different symbols or sensor data readers for each sensor) +- The program does not leave your server and doesn't use any external systems. It runs on the server you install the host on. +- Currently STH supports node.js and Python runners only, we're working on bringing you runners for other languages, with Java and C++ as the first ones. + +We hope this helps you understand the basics of STH and get started with it. If you have any questions or need help, please don't hesitate to contact us! + +Some important links πŸ‘€: + +- All the documentation can be found on our documentation portal [docs.scramjet.org](https://docs.scramjet.org). +- Here you can find the definition of the [Transform Sequence AppContext](./docs/types/interfaces/appcontext.md) +- You can see the [Scramjet Transform Hub API docs here](./docs/interfaces/API-reference.md) +- You can see the [CLI documentation here](./docs/interfaces/CLI-command-reference.md), but `si help` should also be quite effective. +- Don't forget to ⭐ this repo if you like it, `subscribe` to releases and keep visiting us for new versions and updates. +- You can [open an issue - file a bug report or a feature request here](https://github.com/scramjetorg/transform-hub/issues/new/choose) + +--- + +# Development instructions :construction_worker: + +This section contains more detailed installation descriptions, it is intended for developers who would like to contribute or build Transform Hub from source. If you wish to fire up STH quickly and without unnecessary installations, go to Quick Start [installation section](https://github.com/scramjetorg/platform-docs#2-installation), which focuses on starting STH and deploying a sample application. + +In this section we will also show you some useful commands, tools and functionalities that you can use to develop your own programs. + +If you want to help out, we're happy to accept your pull requests. Please follow the below information to start development. + +## Installation :clamp: + +> **Reminder:** This is a development guide. In order to use it, you need to have a Unix/Linux based os (e.g. [Ubuntu](https://ubuntu.com/download/server)), [docker](https://www.docker.com/get-started) and [node.js v16.x](https://nodejs.org/en/) installed. We're working on development guides for Mac and Windows. πŸ”œ + +### Environment setup + +There are several installations you need to perform to get STH up and running, and even more to start developing with us. You may already have some of these below installed, but we will show you how to install them anyway. + +- nvm +- node.js +- yarn +- typescript +- ts-node +- docker +- pip + +Our hub is based on node.js, so you need to install node.js and npm, which is the default package manager for node.js. +To check if you already have Node.js(v16.xx.x) and npm installed, run the following commands in your console: + +```bash +node -v +npm -v +``` + +If none of the above commands is found, **you must proceed with the installation.** + +It is recommended to use a **Node version manager** like [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md) to install Node.js and npm. Do it by running the install script using either of the commands bellow. +> Note: Make sure you replace v0.39.0 with the latest version of nvm. The actual version of nvm to install is [here](https://github.com/nvm-sh/nvm/blob/master/README.md#install--update-script). + +```bash +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash + +wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash +``` + +After executing the above command you should see: + +![nvm_installation_output](./images/nvm_installation_output.png) + +which means that you need to run those two more commands to finish the nvm installation: + +```bash +export NVM_DIR="$HOME/.nvm" +[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" +[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" +``` + +Export lines from above will be saved in your ~/.bashrc file so that you won't need to run it with every session start. + +Now you are ready to **install node.js**, simply type in your console: + +```bash +nvm install --lts # command will install latest LTS Version of Node.js +``` + +> πŸ’‘ **Note**: +> The project is working on Node Long Term Support (LTS) Version, which contains Node Package Manager (NPM) in `^8.15.0` version. +> NodeJS in version `^17.XX.X` will install NPM in version `^8.1.2` and we don't use it right now πŸ˜‰. + +πŸ€“ For more info visit the node.js official [webpage](https://nodejs.org). + +Now you can check installed versions of node.js and npm, run the following commands in your console: + +```bash +$ node -v +v16.13.0 + +$ npm -v +8.15.0 +``` + +**OK!** It looks like you have successfully installed node.js and npm. + +There are two more installations you need to perform, run the following commands in your console one after another: + +```bash +npm install -g yarn # yarn is a package manager +``` + +![installations](./images/installations.png) + +The same as before the installations can be confirmed by checking the installed versions: + +```bash +$ yarn -v +1.22.17 +``` + +**OK!** The installation was successful. πŸŽ‰ πŸŽ† + +STH can run python packages, and for that it needs to have `python` and `pip`. These should be already installed on your system - check with: + +```bash +python3 --version +pip --version +``` + +If they are not present, refer to the official installation guide for [Python](https://wiki.python.org/moin/BeginnersGuide/Download) and [Pip](https://pip.pypa.io/en/stable/installation/). + +> **If you don't want to use Docker, please skip this step!** + +We also work with Docker, but this is optional. Running STH is possible without Docker in the background. If you want to use Docker, you can install it by running the following commands in your console: + +```bash +# Install docker and docker-compose +sudo apt install -y docker.io docker-compose + +# Verify installations by getting versions +sudo docker version +docker-compose version + +# Add you user to the docker group +sudo gpasswd -a $USER docker +``` + +### STH installation + +There are two ways to install STH, either by getting the package from npmjs or by cloning repo. + +**First**, let's install current STH from the npm registry -> [@scramjet/sth](https://www.npmjs.com/package/@scramjet/sth): + +```bash +npm i -g @scramjet/sth # installs the package globally +``` + +![npm_package](./images/npm_package.png) + +When you start STH from npm, no package installations or builds are required. You can start STH right away, with one of those two commands: `scramjet-transform-hub` or `sth`. This was quick, wasn't it?:D + +**Second** way is designed for developers who want to contribute and not *just* use STH. + + Let's clone the transform-hub repo. After that, some more installations will be required. Please copy the following commands to the terminal: + +```bash +git clone https://github.com/scramjetorg/transform-hub.git && \ +cd transform-hub && \ +yarn install && yarn build:all && npm i -g ./dist/cli +``` + +Depending on your machine this may take some time, so it is a perfect time for another hot beverage β˜• or walk πŸšΆπŸΌβ€β™€οΈ or joggling πŸ€Ήβ€β™‚οΈ or push-ups maybe..? no? Then simply wait 🧘. + +In the meantime let me describe for you what is happening in the command you have just pasted into the console: + +- `git clone https://github.com/scramjetorg/transform-hub.git` is cloning STH repository. +- `cd transform-hub` is changing the directory to STH repository. +- `yarn install` is installing all the dependencies of STH. +- `yarn build:all` is building all STH packages, this script includes three other building scripts (yarn build:packages && yarn build:refapps && yarn build:docker). +- `npm i -g ./dist/cli` is installing STH [CLI](https://github.com/scramjetorg/scramjet-cli) as a global command. + +When both the package installation and build are complete, STH should be ready to run. + + + +--- + +## Start STH :checkered_flag: + +STH can be started in multiple ways. The default way is to run it with Docker and on localhost and port number 8000. Choose one of the following: + +If installed by **cloning the repo**: + +```bash +yarn start # Starts Hub after it's been built using script + +yarn start:dev # Starts Hub in development mode +``` +If installed as **npm package**: + +```bash +scramjet-transform-hub # Starts Hub from installed npm package (@scramjet/sth) + +sth # alias for scramjet-transform-hub +``` +You can also start STH without Docker, use the same commands as above but add `--no-docker` flag: + +```bash +yarn start --no-docker + +yarn start:dev --no-docker + +sth --no-docker +``` + +There is a wide range of options that you can start STH with. Please add `--help` or `-h` flag to list all the options: + +```bash +yarn start --help + +yarn start:dev --help + +sth --help +``` + +```shell +$ sth --help +Usage: sth [options] + +Options: + -L, --log-level Specify log level (default: "debug") + -S, --socket-path CSI socket location + -P, --port API port + -H, --hostname API IP + -E, --identify-existing Index existing volumes as sequences (default: false) + -C, --cpm-url + --id + --runner-image Image used by runner + --runner-max-mem Maximum mem used by runner + --prerunner-image Image used by prerunner + --prerunner-max-mem Maximum mem used by prerunner + --expose-host-ip Host IP address that the Runner containers port is mapped to. + --runtime-adapter Run all the instances on the given type of container or on host machine. Possible value: `process`, `docker`, `kubernetes`. Keep in mind that using `process` is UNSAFE FOR RUNNING ARBITRARY CODE. + --sequences-root Only works with --runtime-adapter=process option. Where should ProcessSequenceAdapter save new sequences + -h, --help display help for command +``` + +## Install CLI and execute :white_check_mark: + +CLI installation was already done at the end of the [Installation](#installation-clamp) section, just before starting STH. But it is worth mentioning, that there are two ways to install the CLI: + +- from the source code. In the root folder, **after building**, run the commands: + +```bash +npm i -g ./dist/cli # install CLI globally from the source folder +si --help # show CLI commands +``` + +- from [npm](https://www.npmjs.com/package/@scramjet/cli): + +```bash +npm i -g @scramjet/cli # install CLI globally from npm +si --help # show CLI commands +``` + +![cli_npm](./images/cli_npm.png) + +Running `si --help` command will confirm that the installation went properly and also to see the list of available commands: + +```bash +$ si --help +Current profile: default +Usage: si [options] [command] + +This is a Scramjet Command Line Interface to communicate with Transform Hub and Cloud Platform. + +Options: + -v, --version Display current CLI version + --config Set global configuration profile + --config-path Set global configuration from file + -h, --help display help for command + +Commands: + hub Allows to run programs in different data centers, computers or devices in local network + config|c Config contains default Scramjet Transform Hub (STH) and Scramjet Cloud Platform (SCP) settings + sequence|seq Operations on a Sequence package, consisting of one or more functions executed one after another + instance|inst [command] Operations on the running Sequence + topic Manage data flow through topics operations + completion completion operations + util|u Various utilities + +To find out more about CLI, please check out our docs at https://hub.scramjet.org/docs/cli + +Read more about Scramjet at https://scramjet.org/ πŸš€ +``` + +You can also run CLI commands from the source code using `yarn start:dev:cli` instead of `si`, in this case no installation is needed, eg.: + +```bash +$ yarn start:dev:cli --version +yarn run v1.22.17 + +$ ts-node packages/cli/src/bin/index.ts --version +CLI version: 0.28.1 +``` + +We will use CLI later on to execute the Sequence. If you would like to read more about our command line interface jump right away to [CLI docs](https://github.com/scramjetorg/transform-hub/blob/main/docs/cli) in this repo or explore [CLI Reference](https://docs.scramjet.org/platform/cli-reference) section on our official website [www.scramjet.org](https://scramjet.org/). + +## Build the packages :building_construction: + +All the packages in the project need to be installed and built before they can be used. + +**Install** will create a `node_modules` folder in the root of the project and in each package (dependencies will be installed). + +**Build** will create a `dist` folder in the root of the project and in each package. The folder will contain the files with the code compiled into JavaScript. + +This is how to perform a clean install and build of all the packages: + +```bash +yarn install:clean # this command will perform 'yarn clean && yarn clean:modules && yarn install' at once +yarn build:packages # optionally 'build:all' if you want all dockerfiles. +``` + +![build_clean](./images/clean_install.png) + +> πŸ’‘ **HINT:** For more scripts please see `"scripts: {}"` in main [package.json](package.json#scripts). + +## Docker commands :whale: + +During development, some artifacts may be leftover in docker. See how to clean them: + +```bash +docker ps # list containers +docker volume prune -f # remove all unused local volumes +docker system prune --all -f # remove all unused images not just dangling ones +docker stop $(docker ps -a -q) # stops all running containers +``` + +> **\*(`-f`)** - doesn't prompt confirmation + +## Build Hub on Docker :building_construction: + +Build from current source: + +```bash +cd ./packages/sth/ +yarn build:docker +``` + +## Run Transform Hub in Docker :robot: + +```bash +cd ./packages/sth/ +docker-compose up + +# or run in detached mode +docker-compose up -d +docker-compose logs -f +``` + +To run Hub without docker-compose: + +```bash +docker run \ + -ti \ + --name scramjet-hub \ + --init \ + -p 8000:8000 \ + -v /tmp/:/tmp/ \ + -v /var/run/docker.sock:/var/run/docker.sock \ + scramjetorg/sth:$(jq -r .version package.json) +``` + +## Monorepo commands :pencil: + +We use our own scripts to control our monorepo. Here's a couple of helpful commands, which might be useful during development: + +```bash +./scripts/run-script.js [script] + # Run an npm script in each package that contains that script. +./scripts/run-script.js --scope @scramjet/ + # Run script only in one package +./scripts/run-script.js --workspace packages + # Run script in all packages in workspace + +./scripts/run-script.js --help + # Check this out for additional information +``` + +## Autocomplete :zap: + +Completion script depends on bash-completion so make sure it's already installed by running `type _init_completion`. + +Below command installs completion script in ~/.bashrc. + +```bash +si completion install +``` + +For immediate effect make sure to run `source ~/.bashrc`. Also you can manage completion script by yourself. Running `si completion bash` prints script to the terminal. + +--- + +## Git branching workflow + +There are two important branches: `main` (representing stable version of the +project) and `devel` (where most of the work happens; this is the default +branch). + +- New features, and especially breaking changes, should go to the `devel` branch. +- Bugfixes, especially ones that should be published in a bugfix release, + should be merged into the `main` branch. + - Start such bugfix branches from `main`, and make PRs to `main`. + - After a bugfix is merged, `main` should be merged into `devel` to get the + bugfix there as well. + +The purpose of having two branches is to keep important bugfixes separate from +new features, so that a bugfix version (a release without new features, just +bugfixes) can be published immediately if needed. + +# Sample usage :sunglasses: + +## "Hello World!" sample :wave: + +Further actions will work only if you have properly configured your environment, installed hub and built all the packages. +By this time you should already have all those things done by going through the [Installation](#installation-clamp) section. + +> πŸ’‘ **HINT:** _The following instructions apply to the state of the repository from the `release/0.28.0`._ + +- Run STH: +```bash +yarn start # if cloned STH repo + +sth # if installed STH as npm package +``` +Open a new terminal window **inside transform-hub root** folder and do the following: + +- Send the hello-world Sequence package to the STH and start it: +```bash +si seq send packages/hello-world.tar.gz + +si seq start - # '-' means last uploaded Sequence +si seq start # or you can specify id parameter instead +``` + +Alternatively, you can use ***deploy*** which does 3in1 - it packs (if neccessary), sends and starts the Sequence: + +```bash +si seq deploy packages/hello-world.tar.gz +``` +You should see on the STH terminal that it received the Sequence. + +**Let's connect to the output stream.** Run this command: +```bash +si inst output - # '-' means last uploaded Sequence +si inst output # or you can specify id parameter instead +``` +Note that we need an instance ID, not the Sequence. +If the output from previous command have been cleared on your terminal, you can still list all your Sequences and instances by running the following: +- **List Sequences:** +```bash +si seq list +``` +- **List instances:** +```bash +si inst list +``` +After using instance's ID with `si inst output` command, you should see **"Hello World!"** on your terminal window. + +[See more about streams and curl commands =>](docs/read-more/stream-and-api.md) + +## More samples :books: + +To check out more of our ready-to-go samples, please go to our [Samples repo](https://github.com/scramjetorg/platform-samples) on GitHub, or to our [Samples Hub](https://docs.scramjet.org/platform/samples). + +## Configure your own sample :pencil: + +We have also prepared a template for you to use. You can use it as a base for your own samples πŸ‘‰ [sample template](https://github.com/scramjetorg/platform-docs/tree/main/templates). For this moment we support two variants of template in two programming languages: + +- JavaScript (Node.js) πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-js) +- TypeScript (ts-node) πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-ts) +- Python πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-ts) + +There is also C++ template that is still in development stage: + +# Troubleshooting :collision: + +### **Docker issues:** + +
+ + Error: connect ENOENT /var/run/docker.sock + + +During sending the Sequence compressed package to the host you may come across this error: + +- **_Error: connect ENOENT /var/run/docker.sock_** + + ```bash + 2022-01-13T11:54:26.948Z info (object:Host) New sequence incoming... + 2022-01-13T11:54:26.948Z log (object:DockerSequenceAdapter) DockerSequenceAdapter init. + 2022-01-13T11:54:26.948Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:54:26.949Z log (object:DockerodeDockerHelper) Start pulling image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:54:26.949Z debug (object:Host) Error: connect ENOENT /var/run/docker.sock + at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1161:16) + 2022-01-13T11:54:26.950Z debug (object:Host) Request date: 2022-01-13T11:54:26.948Z, method: POST, url: /api/v1/sequence, status: 422 + ``` + +To solve this issue you need to install docker and docker-compose. You can install them using the following command below or refer to [official Docker instructions](https://docs.docker.com/get-docker/). + + ```bash + sudo apt install -y docker.io docker-compose + ``` + +
+ +
+ + Error: connect EACCES /var/run/docker.sock + + +During sending the Sequence compressed package to the host you may come across this error: + +- **_Error: connect EACCES /var/run/docker.sock_** + + ```bash + 2022-01-13T11:58:00.368Z info (object:Host) New sequence incoming... + 2022-01-13T11:58:00.369Z log (object:DockerSequenceAdapter) DockerSequenceAdapter init. + 2022-01-13T11:58:00.370Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:58:00.375Z log (object:DockerodeDockerHelper) Start pulling image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:58:00.377Z debug (object:Host) Error: connect EACCES /var/run/docker.sock + at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1161:16) + 2022-01-13T11:58:00.379Z debug (object:Host) Request date: 2022-01-13T11:58:00.373Z, method: POST, url: /api/v1/sequence, status: 422 + ``` + +To solve this issue you need to add the current user to the docker group, please use this command: + +```bash +sudo gpasswd -a $USER docker +``` +If the problem persists, restart your PC. +

+ +### **Packages issues:** + +
+ + Errors related to build packages +
+ +If something goes wrong during building packages, any errors occur, please try to run clean build, which will remove all `node_modules` and `dist` directories, after that you try to install and build them again. + +```bash +yarn clean && yarn clean:modules && yarn install && yarn build:all +``` + +
+ +
+ + Errors related to reference-apps package +
+ +Every reference-app package before we run it, needs to: + +- have `node_modules` installed (`yarn install`) +- have `dist` directory created and tar.gz package created (`yarn build:refapps`) + +Remember to install dependencies and build your sample package before compressing it. + +If you create your sample in `packages/reference-apps` folder, you can use the following command to build it: + +```bash +yarn build:refapps +``` + +It will build all the packages in the `packages/reference-apps` folder. + +

+ +> **πŸ’‘ HINT:** Have a look at the root `package.json`, there is the `scripts` section, which contains the list of all the scripts you can run with yarn. You may find them useful. + +Log an issue/bug every time you encounter a problem or find a bug. Maybe you will also find that some feature is missing? + +- [bug report](https://github.com/scramjetorg/transform-hub/issues/new?assignees=&labels=&template=bug_report.md&title=) +- [feature request](https://github.com/scramjetorg/transform-hub/issues/new?assignees=&labels=&template=feature_request.md&title=) + +--- + +# Start your 30 days trial now! :partying_face: + +[![Register](https://user-images.githubusercontent.com/63969229/208955381-676702f3-2af1-4f50-bd0c-f6b3660c26f0.png)](https://scramjet.org/) + +--- + +# License and contributions :page_with_curl: + +This project is dual licensed under the AGPL-3.0 and MIT licenses. Parts of the project that are linked with your programs are MIT licensed, the rest is AGPL. + +We accept valid contributions and we will be publishing a more specific project road map so contributors can propose features and also help us implement them. We kindly ask you that contributed commits are Signed-Off `git commit --sign-off`. + +We provide support for contributions via test cases. If you expect a certain type of workflow to be officially supported, please specify and implement a test case in `Gherkin` format in [`bdd` directory](./bdd). + +More info about bdd tests can be found [here](./bdd/README.md). + +--- + +# Help wanted :information_desk_person: + +The project need's your help! There's lots of work to do and we have a lot of plans. If you want to help and be part of the Scramjet team, please reach out to us, [on discord](https://scr.je/join-community-mg1) or email us: [opensource@scramjet.org](opensource@scramjet.org). + +--- + +# Donation :money_with_wings: + +Do you like this project? It helped you to reduce time spent on delivering your solution? You are welcome to buy us a coffee β˜• + +- [You can sponsor us on github](https://github.com/sponsors/scramjetorg) + +- There's also a Paypal donation link if you prefer that: + +[![paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=7F7V65C43EBMW) diff --git a/docs/module-loader/modules.md b/docs/module-loader/modules.md new file mode 100644 index 000000000..c90623f7e --- /dev/null +++ b/docs/module-loader/modules.md @@ -0,0 +1,49 @@ +[@scramjet/module-loader](README.md) / Exports + +# @scramjet/module-loader + +## Table of contents + +### Functions + +- [loadModule](modules.md#loadmodule) + +### Variables + +- [logger](modules.md#logger) + +## Functions + +### loadModule + +β–Έ **loadModule**<`T`\>(`opts`): `Promise`<`T`\> + +#### Type parameters + +| Name | +| :------ | +| `T` | + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `opts` | `ModuleLoaderOpts` | + +#### Returns + +`Promise`<`T`\> + +#### Defined in + +[index.ts:5](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/module-loader/src/index.ts#L5) + +## Variables + +### logger + +β€’ `Const` **logger**: `ObjLogger` + +#### Defined in + +[index.ts:4](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/module-loader/src/index.ts#L4) diff --git a/docs/monitoring-server/README.md b/docs/monitoring-server/README.md new file mode 100644 index 000000000..2fb5e365b --- /dev/null +++ b/docs/monitoring-server/README.md @@ -0,0 +1,743 @@ +@scramjet/monitoring-server / [Exports](modules.md) + +

Scramjet Transform Hub

+

+ GitHub license + STH version + GitHub stars + npm + Discord + + Donate + +

+

⭐ Star us on GitHub β€” it motivates us a lot! πŸš€

+

+ Scramjet Transform Hub Logo +

+ +# The Idea + +Scramjet Transform Hub is a cool way to run programs on your computer or in the cloud. It's easy to use, and it supports lots of different types of programs. + +The Hub is the heart of Scramjet Cloud Platform, a serverless data processing system that allows you to easily deploy, run and interconnect programs that process repetitive data tasks in long-running processes. STH can be run just as well on a Raspberry Pi as it can on a massive 128 core Epyc bare metal server. It installs in one simple command and deploys your app to processes, Docker containers or Kubernetes clusters just as easily... Oh, and the apps are incredibly simple to write too, here's a slack posting sample: + +```js +export (input: Readable) => async function*() { + for (const post of input) { + yield await slack.note(`New post titled "${post.title}" from ${post.author}`); + } +} +``` + +It currently supports Node.js and Python based apps, but Java and other languages are coming. + +[Get Scramjet Transform Hub straight from NPM](https://www.npmjs.com/package/@scramjet/sth) + +If you don't have a server ready or want to run something from the cloud itself, do check out our website and try the hosted version of [Scramjet Cloud Platform](https://scramjet.org/). + +--- + +# Table of contents + +- [Introduction :handshake:](#introduction-handshake) +- [Usage :bulb:](#usage-bulb) +- [The basics :abcd:](#the-basics-abcd) +- [Development instructions :construction_worker:](#development-instructions-construction_worker) + - [Installation :clamp:](#installation-clamp) + - [Environment setup](#environment-setup) + - [STH installation](#sth-installation) + - [Start STH :checkered_flag:](#start-sth-checkered_flag) + - [Install CLI and execute :white_check_mark:](#install-cli-and-execute-white_check_mark) + - [Build the packages :building_construction:](#build-the-packages-building_construction) + - [Docker commands :whale:](#docker-commands-whale) + - [Build Hub on Docker :building_construction:](#build-hub-on-docker-building_construction) + - [Run Transform Hub in Docker :robot:](#run-transform-hub-in-docker-robot) + - [Monorepo commands :pencil:](#monorepo-commands-pencil) + - [Autocomplete :zap:](#autocomplete-zap) + - [Git branching workflow](#git-branching-workflow) +- [Sample usage :sunglasses:](#sample-usage-sunglasses) + - ["Hello Alice" sample :wave:](#hello-alice-sample-wave) + - [More samples :books:](#more-samples-books) + - [Configure your own sample :pencil:](#configure-your-own-sample-pencil) +- [Troubleshooting :collision:](#troubleshooting-collision) +- [Start your 30 day trial :partying_face:](#start-your-30-days-trial-now-partying_face) +- [License and contributions :page_with_curl:](#license-and-contributions-page_with_curl) +- [Help wanted :information_desk_person:](#help-wanted-information_desk_person) +- [Donation :money_with_wings:](#donation-money_with_wings) + +--- + +# Introduction :handshake: + +This is a development repo for Scramjet Transform Hub, a container supervisor that allows deployment, execution and monitoring of any application based on a simple interface. + +This repository contains the source code for [Scramjet Transform Hub](https://docs.scramjet.org/platform/transform-hub). + +If you want to read more about the usage please go to [Quick Start](https://github.com/scramjetorg/platform-docs) repo. + +You may also be interested in our [Scramjet Cloud Platform](https://docs.scramjet.org/platform) that is based on Transform-Hub. + +What's more, this repository is intended for developers who would like to: + +- participate in the Scramjet Transform Hub development community +- register feature requests, issues and PRs for this product +- build Transform Hub from source +- dive into code and see how it works + +For developers who would like to simply use Scramjet Transform Hub to run data processing applications, we recommend following resources: + +![gh_logo](./images/gh_logo.png) [Repository with Introductory README and templates](https://github.com/scramjetorg/platform-docs) + +![gh_logo](./images/gh_logo.png) [Repository with samples](https://github.com/scramjetorg/platform-samples) + +![npm_logo](./images/npm_logo.png) [NPM page for main scramjet/sth package](https://www.npmjs.com/package/@scramjet/sth) + +![npm_logo](./images/npm_logo.png) [NPM page for scramjet/cli package](https://www.npmjs.com/package/@scramjet/cli) + +--- + +# Usage :bulb: + +The easiest way to start Transform Hub is to run the follow (you'll need node.js and npm installed): + +```bash +npx @scramjet/sth --runtime-adapter= +``` + +Scramjet Transform Hub is a platform that helps you run programs on local and remote machines. You can start programs in the background or connect to them and see their output. You can also pipe your data to the program even if it is running on a different machine. You can install this platform in different places like AWS, Google Cloud, Azure or even your own computer. + +There's no limit to what you can use it for. Do you want a stock checker? A chatbot? Maybe you'd like to automate your home? Retrieve sensor data? Maybe you have a lot of data and want to transfer and wrangle it? You have a database of cities and you'd like to enrich your data? You do machine learning and you want to train your set while the data is fetched in real-time? Or perhaps you want to use it for something else and ask us if that's a good use? Ask us [via email](mailto:get@scramjet.org) πŸ“§ or hop on our [Scramjet Discord ![discord](./images/discord.png)](https://scr.je/join-community-mg1) and we'll be happy to talk with you. + +--- + +# The basics :abcd: + +**NOTE:** This is the STH development repo and in order to use it, you need to have **Unix/Linux** based operating system, for example [Ubuntu](https://ubuntu.com/download/server) or [Windows with WSL2](https://learn.microsoft.com/en-us/windows/wsl/install). We also use [docker](https://www.docker.com/get-started) and [node.js v16.x](https://nodejs.org/en/). We're working on development guides for Mac and Windows. πŸ”œ + +Scramjet Transform Hub allows you to deploy and execute programs that you build and develop. As mentioned above, you can run any program you like, but you need to know a couple of important things: + +- The program should consist of a function or an array of functions, such a program is called a **Transform Sequence**. +- The Sequence will be executed within a separate docker instance (πŸ”œ we're working on other execution environment integrations - help will be appreciated 🦾). +- The Sequence function will receive a stream as input in the first argument - you can send the data to it via the command `si instance input`. +- If your Sequence contains more than one function, then the output from the first function is passed to the next one. Also, the first function in Sequence receives the input from API. +- The functions can also be Generators and iterate over input data in `node` and `python` versions. This is the simplest way to process data, but it won't leverage asynchronous processing. Check out our frameworks to run your transforms faster [Scramjet Framework](https://github.com/scramjetorg/scramjet#scramjet-framework) +- The last (or the only) function in Sequence can return a `Promise` or a `Stream` - based on this, STH will know when processing is done. +- Once the returned `Promise` or `future` is resolved, or the `Stream` is ended, STH will gracefully stop the Sequence and remove its container. +- You can communicate with the server via API, command line client `si` which we wrote for your convenience. +- The Sequence is called with an AppContext as `this` or `self`, a class that allows you to communicate back from the Sequence: send logs, provide health info, send and receive events from the API or CLI. +- You can run your Sequence multiple times with different arguments (like for instance currency tickers with different symbols or sensor data readers for each sensor) +- The program does not leave your server and doesn't use any external systems. It runs on the server you install the host on. +- Currently STH supports node.js and Python runners only, we're working on bringing you runners for other languages, with Java and C++ as the first ones. + +We hope this helps you understand the basics of STH and get started with it. If you have any questions or need help, please don't hesitate to contact us! + +Some important links πŸ‘€: + +- All the documentation can be found on our documentation portal [docs.scramjet.org](https://docs.scramjet.org). +- Here you can find the definition of the [Transform Sequence AppContext](./docs/types/interfaces/appcontext.md) +- You can see the [Scramjet Transform Hub API docs here](./docs/interfaces/API-reference.md) +- You can see the [CLI documentation here](./docs/interfaces/CLI-command-reference.md), but `si help` should also be quite effective. +- Don't forget to ⭐ this repo if you like it, `subscribe` to releases and keep visiting us for new versions and updates. +- You can [open an issue - file a bug report or a feature request here](https://github.com/scramjetorg/transform-hub/issues/new/choose) + +--- + +# Development instructions :construction_worker: + +This section contains more detailed installation descriptions, it is intended for developers who would like to contribute or build Transform Hub from source. If you wish to fire up STH quickly and without unnecessary installations, go to Quick Start [installation section](https://github.com/scramjetorg/platform-docs#2-installation), which focuses on starting STH and deploying a sample application. + +In this section we will also show you some useful commands, tools and functionalities that you can use to develop your own programs. + +If you want to help out, we're happy to accept your pull requests. Please follow the below information to start development. + +## Installation :clamp: + +> **Reminder:** This is a development guide. In order to use it, you need to have a Unix/Linux based os (e.g. [Ubuntu](https://ubuntu.com/download/server)), [docker](https://www.docker.com/get-started) and [node.js v16.x](https://nodejs.org/en/) installed. We're working on development guides for Mac and Windows. πŸ”œ + +### Environment setup + +There are several installations you need to perform to get STH up and running, and even more to start developing with us. You may already have some of these below installed, but we will show you how to install them anyway. + +- nvm +- node.js +- yarn +- typescript +- ts-node +- docker +- pip + +Our hub is based on node.js, so you need to install node.js and npm, which is the default package manager for node.js. +To check if you already have Node.js(v16.xx.x) and npm installed, run the following commands in your console: + +```bash +node -v +npm -v +``` + +If none of the above commands is found, **you must proceed with the installation.** + +It is recommended to use a **Node version manager** like [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md) to install Node.js and npm. Do it by running the install script using either of the commands bellow. +> Note: Make sure you replace v0.39.0 with the latest version of nvm. The actual version of nvm to install is [here](https://github.com/nvm-sh/nvm/blob/master/README.md#install--update-script). + +```bash +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash + +wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash +``` + +After executing the above command you should see: + +![nvm_installation_output](./images/nvm_installation_output.png) + +which means that you need to run those two more commands to finish the nvm installation: + +```bash +export NVM_DIR="$HOME/.nvm" +[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" +[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" +``` + +Export lines from above will be saved in your ~/.bashrc file so that you won't need to run it with every session start. + +Now you are ready to **install node.js**, simply type in your console: + +```bash +nvm install --lts # command will install latest LTS Version of Node.js +``` + +> πŸ’‘ **Note**: +> The project is working on Node Long Term Support (LTS) Version, which contains Node Package Manager (NPM) in `^8.15.0` version. +> NodeJS in version `^17.XX.X` will install NPM in version `^8.1.2` and we don't use it right now πŸ˜‰. + +πŸ€“ For more info visit the node.js official [webpage](https://nodejs.org). + +Now you can check installed versions of node.js and npm, run the following commands in your console: + +```bash +$ node -v +v16.13.0 + +$ npm -v +8.15.0 +``` + +**OK!** It looks like you have successfully installed node.js and npm. + +There are two more installations you need to perform, run the following commands in your console one after another: + +```bash +npm install -g yarn # yarn is a package manager +``` + +![installations](./images/installations.png) + +The same as before the installations can be confirmed by checking the installed versions: + +```bash +$ yarn -v +1.22.17 +``` + +**OK!** The installation was successful. πŸŽ‰ πŸŽ† + +STH can run python packages, and for that it needs to have `python` and `pip`. These should be already installed on your system - check with: + +```bash +python3 --version +pip --version +``` + +If they are not present, refer to the official installation guide for [Python](https://wiki.python.org/moin/BeginnersGuide/Download) and [Pip](https://pip.pypa.io/en/stable/installation/). + +> **If you don't want to use Docker, please skip this step!** + +We also work with Docker, but this is optional. Running STH is possible without Docker in the background. If you want to use Docker, you can install it by running the following commands in your console: + +```bash +# Install docker and docker-compose +sudo apt install -y docker.io docker-compose + +# Verify installations by getting versions +sudo docker version +docker-compose version + +# Add you user to the docker group +sudo gpasswd -a $USER docker +``` + +### STH installation + +There are two ways to install STH, either by getting the package from npmjs or by cloning repo. + +**First**, let's install current STH from the npm registry -> [@scramjet/sth](https://www.npmjs.com/package/@scramjet/sth): + +```bash +npm i -g @scramjet/sth # installs the package globally +``` + +![npm_package](./images/npm_package.png) + +When you start STH from npm, no package installations or builds are required. You can start STH right away, with one of those two commands: `scramjet-transform-hub` or `sth`. This was quick, wasn't it?:D + +**Second** way is designed for developers who want to contribute and not *just* use STH. + + Let's clone the transform-hub repo. After that, some more installations will be required. Please copy the following commands to the terminal: + +```bash +git clone https://github.com/scramjetorg/transform-hub.git && \ +cd transform-hub && \ +yarn install && yarn build:all && npm i -g ./dist/cli +``` + +Depending on your machine this may take some time, so it is a perfect time for another hot beverage β˜• or walk πŸšΆπŸΌβ€β™€οΈ or joggling πŸ€Ήβ€β™‚οΈ or push-ups maybe..? no? Then simply wait 🧘. + +In the meantime let me describe for you what is happening in the command you have just pasted into the console: + +- `git clone https://github.com/scramjetorg/transform-hub.git` is cloning STH repository. +- `cd transform-hub` is changing the directory to STH repository. +- `yarn install` is installing all the dependencies of STH. +- `yarn build:all` is building all STH packages, this script includes three other building scripts (yarn build:packages && yarn build:refapps && yarn build:docker). +- `npm i -g ./dist/cli` is installing STH [CLI](https://github.com/scramjetorg/scramjet-cli) as a global command. + +When both the package installation and build are complete, STH should be ready to run. + + + +--- + +## Start STH :checkered_flag: + +STH can be started in multiple ways. The default way is to run it with Docker and on localhost and port number 8000. Choose one of the following: + +If installed by **cloning the repo**: + +```bash +yarn start # Starts Hub after it's been built using script + +yarn start:dev # Starts Hub in development mode +``` +If installed as **npm package**: + +```bash +scramjet-transform-hub # Starts Hub from installed npm package (@scramjet/sth) + +sth # alias for scramjet-transform-hub +``` +You can also start STH without Docker, use the same commands as above but add `--no-docker` flag: + +```bash +yarn start --no-docker + +yarn start:dev --no-docker + +sth --no-docker +``` + +There is a wide range of options that you can start STH with. Please add `--help` or `-h` flag to list all the options: + +```bash +yarn start --help + +yarn start:dev --help + +sth --help +``` + +```shell +$ sth --help +Usage: sth [options] + +Options: + -L, --log-level Specify log level (default: "debug") + -S, --socket-path CSI socket location + -P, --port API port + -H, --hostname API IP + -E, --identify-existing Index existing volumes as sequences (default: false) + -C, --cpm-url + --id + --runner-image Image used by runner + --runner-max-mem Maximum mem used by runner + --prerunner-image Image used by prerunner + --prerunner-max-mem Maximum mem used by prerunner + --expose-host-ip Host IP address that the Runner containers port is mapped to. + --runtime-adapter Run all the instances on the given type of container or on host machine. Possible value: `process`, `docker`, `kubernetes`. Keep in mind that using `process` is UNSAFE FOR RUNNING ARBITRARY CODE. + --sequences-root Only works with --runtime-adapter=process option. Where should ProcessSequenceAdapter save new sequences + -h, --help display help for command +``` + +## Install CLI and execute :white_check_mark: + +CLI installation was already done at the end of the [Installation](#installation-clamp) section, just before starting STH. But it is worth mentioning, that there are two ways to install the CLI: + +- from the source code. In the root folder, **after building**, run the commands: + +```bash +npm i -g ./dist/cli # install CLI globally from the source folder +si --help # show CLI commands +``` + +- from [npm](https://www.npmjs.com/package/@scramjet/cli): + +```bash +npm i -g @scramjet/cli # install CLI globally from npm +si --help # show CLI commands +``` + +![cli_npm](./images/cli_npm.png) + +Running `si --help` command will confirm that the installation went properly and also to see the list of available commands: + +```bash +$ si --help +Current profile: default +Usage: si [options] [command] + +This is a Scramjet Command Line Interface to communicate with Transform Hub and Cloud Platform. + +Options: + -v, --version Display current CLI version + --config Set global configuration profile + --config-path Set global configuration from file + -h, --help display help for command + +Commands: + hub Allows to run programs in different data centers, computers or devices in local network + config|c Config contains default Scramjet Transform Hub (STH) and Scramjet Cloud Platform (SCP) settings + sequence|seq Operations on a Sequence package, consisting of one or more functions executed one after another + instance|inst [command] Operations on the running Sequence + topic Manage data flow through topics operations + completion completion operations + util|u Various utilities + +To find out more about CLI, please check out our docs at https://hub.scramjet.org/docs/cli + +Read more about Scramjet at https://scramjet.org/ πŸš€ +``` + +You can also run CLI commands from the source code using `yarn start:dev:cli` instead of `si`, in this case no installation is needed, eg.: + +```bash +$ yarn start:dev:cli --version +yarn run v1.22.17 + +$ ts-node packages/cli/src/bin/index.ts --version +CLI version: 0.28.1 +``` + +We will use CLI later on to execute the Sequence. If you would like to read more about our command line interface jump right away to [CLI docs](https://github.com/scramjetorg/transform-hub/blob/main/docs/cli) in this repo or explore [CLI Reference](https://docs.scramjet.org/platform/cli-reference) section on our official website [www.scramjet.org](https://scramjet.org/). + +## Build the packages :building_construction: + +All the packages in the project need to be installed and built before they can be used. + +**Install** will create a `node_modules` folder in the root of the project and in each package (dependencies will be installed). + +**Build** will create a `dist` folder in the root of the project and in each package. The folder will contain the files with the code compiled into JavaScript. + +This is how to perform a clean install and build of all the packages: + +```bash +yarn install:clean # this command will perform 'yarn clean && yarn clean:modules && yarn install' at once +yarn build:packages # optionally 'build:all' if you want all dockerfiles. +``` + +![build_clean](./images/clean_install.png) + +> πŸ’‘ **HINT:** For more scripts please see `"scripts: {}"` in main [package.json](package.json#scripts). + +## Docker commands :whale: + +During development, some artifacts may be leftover in docker. See how to clean them: + +```bash +docker ps # list containers +docker volume prune -f # remove all unused local volumes +docker system prune --all -f # remove all unused images not just dangling ones +docker stop $(docker ps -a -q) # stops all running containers +``` + +> **\*(`-f`)** - doesn't prompt confirmation + +## Build Hub on Docker :building_construction: + +Build from current source: + +```bash +cd ./packages/sth/ +yarn build:docker +``` + +## Run Transform Hub in Docker :robot: + +```bash +cd ./packages/sth/ +docker-compose up + +# or run in detached mode +docker-compose up -d +docker-compose logs -f +``` + +To run Hub without docker-compose: + +```bash +docker run \ + -ti \ + --name scramjet-hub \ + --init \ + -p 8000:8000 \ + -v /tmp/:/tmp/ \ + -v /var/run/docker.sock:/var/run/docker.sock \ + scramjetorg/sth:$(jq -r .version package.json) +``` + +## Monorepo commands :pencil: + +We use our own scripts to control our monorepo. Here's a couple of helpful commands, which might be useful during development: + +```bash +./scripts/run-script.js [script] + # Run an npm script in each package that contains that script. +./scripts/run-script.js --scope @scramjet/ + # Run script only in one package +./scripts/run-script.js --workspace packages + # Run script in all packages in workspace + +./scripts/run-script.js --help + # Check this out for additional information +``` + +## Autocomplete :zap: + +Completion script depends on bash-completion so make sure it's already installed by running `type _init_completion`. + +Below command installs completion script in ~/.bashrc. + +```bash +si completion install +``` + +For immediate effect make sure to run `source ~/.bashrc`. Also you can manage completion script by yourself. Running `si completion bash` prints script to the terminal. + +--- + +## Git branching workflow + +There are two important branches: `main` (representing stable version of the +project) and `devel` (where most of the work happens; this is the default +branch). + +- New features, and especially breaking changes, should go to the `devel` branch. +- Bugfixes, especially ones that should be published in a bugfix release, + should be merged into the `main` branch. + - Start such bugfix branches from `main`, and make PRs to `main`. + - After a bugfix is merged, `main` should be merged into `devel` to get the + bugfix there as well. + +The purpose of having two branches is to keep important bugfixes separate from +new features, so that a bugfix version (a release without new features, just +bugfixes) can be published immediately if needed. + +# Sample usage :sunglasses: + +## "Hello World!" sample :wave: + +Further actions will work only if you have properly configured your environment, installed hub and built all the packages. +By this time you should already have all those things done by going through the [Installation](#installation-clamp) section. + +> πŸ’‘ **HINT:** _The following instructions apply to the state of the repository from the `release/0.28.0`._ + +- Run STH: +```bash +yarn start # if cloned STH repo + +sth # if installed STH as npm package +``` +Open a new terminal window **inside transform-hub root** folder and do the following: + +- Send the hello-world Sequence package to the STH and start it: +```bash +si seq send packages/hello-world.tar.gz + +si seq start - # '-' means last uploaded Sequence +si seq start # or you can specify id parameter instead +``` + +Alternatively, you can use ***deploy*** which does 3in1 - it packs (if neccessary), sends and starts the Sequence: + +```bash +si seq deploy packages/hello-world.tar.gz +``` +You should see on the STH terminal that it received the Sequence. + +**Let's connect to the output stream.** Run this command: +```bash +si inst output - # '-' means last uploaded Sequence +si inst output # or you can specify id parameter instead +``` +Note that we need an instance ID, not the Sequence. +If the output from previous command have been cleared on your terminal, you can still list all your Sequences and instances by running the following: +- **List Sequences:** +```bash +si seq list +``` +- **List instances:** +```bash +si inst list +``` +After using instance's ID with `si inst output` command, you should see **"Hello World!"** on your terminal window. + +[See more about streams and curl commands =>](docs/read-more/stream-and-api.md) + +## More samples :books: + +To check out more of our ready-to-go samples, please go to our [Samples repo](https://github.com/scramjetorg/platform-samples) on GitHub, or to our [Samples Hub](https://docs.scramjet.org/platform/samples). + +## Configure your own sample :pencil: + +We have also prepared a template for you to use. You can use it as a base for your own samples πŸ‘‰ [sample template](https://github.com/scramjetorg/platform-docs/tree/main/templates). For this moment we support two variants of template in two programming languages: + +- JavaScript (Node.js) πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-js) +- TypeScript (ts-node) πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-ts) +- Python πŸ‘‰ [template](https://github.com/scramjetorg/platform-docs/tree/main/templates/template-ts) + +There is also C++ template that is still in development stage: + +# Troubleshooting :collision: + +### **Docker issues:** + +
+ + Error: connect ENOENT /var/run/docker.sock + + +During sending the Sequence compressed package to the host you may come across this error: + +- **_Error: connect ENOENT /var/run/docker.sock_** + + ```bash + 2022-01-13T11:54:26.948Z info (object:Host) New sequence incoming... + 2022-01-13T11:54:26.948Z log (object:DockerSequenceAdapter) DockerSequenceAdapter init. + 2022-01-13T11:54:26.948Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:54:26.949Z log (object:DockerodeDockerHelper) Start pulling image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:54:26.949Z debug (object:Host) Error: connect ENOENT /var/run/docker.sock + at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1161:16) + 2022-01-13T11:54:26.950Z debug (object:Host) Request date: 2022-01-13T11:54:26.948Z, method: POST, url: /api/v1/sequence, status: 422 + ``` + +To solve this issue you need to install docker and docker-compose. You can install them using the following command below or refer to [official Docker instructions](https://docs.docker.com/get-docker/). + + ```bash + sudo apt install -y docker.io docker-compose + ``` + +
+ +
+ + Error: connect EACCES /var/run/docker.sock + + +During sending the Sequence compressed package to the host you may come across this error: + +- **_Error: connect EACCES /var/run/docker.sock_** + + ```bash + 2022-01-13T11:58:00.368Z info (object:Host) New sequence incoming... + 2022-01-13T11:58:00.369Z log (object:DockerSequenceAdapter) DockerSequenceAdapter init. + 2022-01-13T11:58:00.370Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:58:00.375Z log (object:DockerodeDockerHelper) Start pulling image scramjetorg/pre-runner:0.14.0 + 2022-01-13T11:58:00.377Z debug (object:Host) Error: connect EACCES /var/run/docker.sock + at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1161:16) + 2022-01-13T11:58:00.379Z debug (object:Host) Request date: 2022-01-13T11:58:00.373Z, method: POST, url: /api/v1/sequence, status: 422 + ``` + +To solve this issue you need to add the current user to the docker group, please use this command: + +```bash +sudo gpasswd -a $USER docker +``` +If the problem persists, restart your PC. +

+ +### **Packages issues:** + +
+ + Errors related to build packages +
+ +If something goes wrong during building packages, any errors occur, please try to run clean build, which will remove all `node_modules` and `dist` directories, after that you try to install and build them again. + +```bash +yarn clean && yarn clean:modules && yarn install && yarn build:all +``` + +
+ +
+ + Errors related to reference-apps package +
+ +Every reference-app package before we run it, needs to: + +- have `node_modules` installed (`yarn install`) +- have `dist` directory created and tar.gz package created (`yarn build:refapps`) + +Remember to install dependencies and build your sample package before compressing it. + +If you create your sample in `packages/reference-apps` folder, you can use the following command to build it: + +```bash +yarn build:refapps +``` + +It will build all the packages in the `packages/reference-apps` folder. + +

+ +> **πŸ’‘ HINT:** Have a look at the root `package.json`, there is the `scripts` section, which contains the list of all the scripts you can run with yarn. You may find them useful. + +Log an issue/bug every time you encounter a problem or find a bug. Maybe you will also find that some feature is missing? + +- [bug report](https://github.com/scramjetorg/transform-hub/issues/new?assignees=&labels=&template=bug_report.md&title=) +- [feature request](https://github.com/scramjetorg/transform-hub/issues/new?assignees=&labels=&template=feature_request.md&title=) + +--- + +# Start your 30 days trial now! :partying_face: + +[![Register](https://user-images.githubusercontent.com/63969229/208955381-676702f3-2af1-4f50-bd0c-f6b3660c26f0.png)](https://scramjet.org/) + +--- + +# License and contributions :page_with_curl: + +This project is dual licensed under the AGPL-3.0 and MIT licenses. Parts of the project that are linked with your programs are MIT licensed, the rest is AGPL. + +We accept valid contributions and we will be publishing a more specific project road map so contributors can propose features and also help us implement them. We kindly ask you that contributed commits are Signed-Off `git commit --sign-off`. + +We provide support for contributions via test cases. If you expect a certain type of workflow to be officially supported, please specify and implement a test case in `Gherkin` format in [`bdd` directory](./bdd). + +More info about bdd tests can be found [here](./bdd/README.md). + +--- + +# Help wanted :information_desk_person: + +The project need's your help! There's lots of work to do and we have a lot of plans. If you want to help and be part of the Scramjet team, please reach out to us, [on discord](https://scr.je/join-community-mg1) or email us: [opensource@scramjet.org](opensource@scramjet.org). + +--- + +# Donation :money_with_wings: + +Do you like this project? It helped you to reduce time spent on delivering your solution? You are welcome to buy us a coffee β˜• + +- [You can sponsor us on github](https://github.com/sponsors/scramjetorg) + +- There's also a Paypal donation link if you prefer that: + +[![paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=7F7V65C43EBMW) diff --git a/docs/monitoring-server/classes/MonitoringServer.md b/docs/monitoring-server/classes/MonitoringServer.md new file mode 100644 index 000000000..3d2fcca6e --- /dev/null +++ b/docs/monitoring-server/classes/MonitoringServer.md @@ -0,0 +1,66 @@ +[@scramjet/monitoring-server](../README.md) / [Exports](../modules.md) / MonitoringServer + +# Class: MonitoringServer + +## Implements + +- `IMonitoringServer` + +## Table of contents + +### Constructors + +- [constructor](MonitoringServer.md#constructor) + +### Methods + +- [handleHealtzRequest](MonitoringServer.md#handlehealtzrequest) +- [start](MonitoringServer.md#start) + +## Constructors + +### constructor + +β€’ **new MonitoringServer**(`options`) + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `options` | `MonitoringServerOptions` | + +#### Defined in + +[monitoring-server.ts:9](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/monitoring-server/src/monitoring-server.ts#L9) + +## Methods + +### handleHealtzRequest + +β–Έ **handleHealtzRequest**(): `Promise`<`boolean`\> + +#### Returns + +`Promise`<`boolean`\> + +#### Defined in + +[monitoring-server.ts:31](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/monitoring-server/src/monitoring-server.ts#L31) + +___ + +### start + +β–Έ **start**(): `Promise`<`MonitoringServerConfig`\> + +#### Returns + +`Promise`<`MonitoringServerConfig`\> + +#### Implementation of + +IMonitoringServer.start + +#### Defined in + +[monitoring-server.ts:35](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/monitoring-server/src/monitoring-server.ts#L35) diff --git a/docs/monitoring-server/modules.md b/docs/monitoring-server/modules.md new file mode 100644 index 000000000..dab3f4998 --- /dev/null +++ b/docs/monitoring-server/modules.md @@ -0,0 +1,9 @@ +[@scramjet/monitoring-server](README.md) / Exports + +# @scramjet/monitoring-server + +## Table of contents + +### Classes + +- [MonitoringServer](classes/MonitoringServer.md) diff --git a/docs/symbols/enums/DisconnectHubErrors.md b/docs/symbols/enums/DisconnectHubErrors.md new file mode 100644 index 000000000..51a678376 --- /dev/null +++ b/docs/symbols/enums/DisconnectHubErrors.md @@ -0,0 +1,96 @@ +[@scramjet/symbols](../README.md) / [Exports](../modules.md) / DisconnectHubErrors + +# Enumeration: DisconnectHubErrors + +## Table of contents + +### Enumeration Members + +- [ALREADY\_DISCONNECTED](DisconnectHubErrors.md#already_disconnected) +- [BAD\_REQUEST](DisconnectHubErrors.md#bad_request) +- [CONFLICT](DisconnectHubErrors.md#conflict) +- [CONNECTED](DisconnectHubErrors.md#connected) +- [ID\_NOT\_PROVIDED](DisconnectHubErrors.md#id_not_provided) +- [KEY\_NOT\_FOUND](DisconnectHubErrors.md#key_not_found) +- [NATIVE\_HUB](DisconnectHubErrors.md#native_hub) +- [STH\_NOT\_FOUND](DisconnectHubErrors.md#sth_not_found) + +## Enumeration Members + +### ALREADY\_DISCONNECTED + +β€’ **ALREADY\_DISCONNECTED** = ``7`` + +#### Defined in + +[disconnect-error-codes.ts:9](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L9) + +___ + +### BAD\_REQUEST + +β€’ **BAD\_REQUEST** = ``2`` + +#### Defined in + +[disconnect-error-codes.ts:4](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L4) + +___ + +### CONFLICT + +β€’ **CONFLICT** = ``3`` + +#### Defined in + +[disconnect-error-codes.ts:5](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L5) + +___ + +### CONNECTED + +β€’ **CONNECTED** = ``6`` + +#### Defined in + +[disconnect-error-codes.ts:8](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L8) + +___ + +### ID\_NOT\_PROVIDED + +β€’ **ID\_NOT\_PROVIDED** = ``4`` + +#### Defined in + +[disconnect-error-codes.ts:6](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L6) + +___ + +### KEY\_NOT\_FOUND + +β€’ **KEY\_NOT\_FOUND** = ``1`` + +#### Defined in + +[disconnect-error-codes.ts:3](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L3) + +___ + +### NATIVE\_HUB + +β€’ **NATIVE\_HUB** = ``5`` + +#### Defined in + +[disconnect-error-codes.ts:7](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L7) + +___ + +### STH\_NOT\_FOUND + +β€’ **STH\_NOT\_FOUND** = ``0`` + +#### Defined in + +[disconnect-error-codes.ts:2](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/symbols/src/disconnect-error-codes.ts#L2) diff --git a/docs/symbols/modules.md b/docs/symbols/modules.md index cf55be955..58262d362 100644 --- a/docs/symbols/modules.md +++ b/docs/symbols/modules.md @@ -9,6 +9,7 @@ - [APIErrorCode](enums/APIErrorCode.md) - [CPMMessageCode](enums/CPMMessageCode.md) - [CommunicationChannel](enums/CommunicationChannel.md) +- [DisconnectHubErrors](enums/DisconnectHubErrors.md) - [HostHeaders](enums/HostHeaders.md) - [InstanceMessageCode](enums/InstanceMessageCode.md) - [OpRecordCode](enums/OpRecordCode.md) diff --git a/docs/types/interfaces/IMonitoringServer.md b/docs/types/interfaces/IMonitoringServer.md new file mode 100644 index 000000000..525fcdf5e --- /dev/null +++ b/docs/types/interfaces/IMonitoringServer.md @@ -0,0 +1,23 @@ +[@scramjet/types](../README.md) / [Exports](../modules.md) / IMonitoringServer + +# Interface: IMonitoringServer + +## Table of contents + +### Methods + +- [start](IMonitoringServer.md#start) + +## Methods + +### start + +β–Έ **start**(): `Promise`<[`MonitoringServerConfig`](../modules.md#monitoringserverconfig)\> + +#### Returns + +`Promise`<[`MonitoringServerConfig`](../modules.md#monitoringserverconfig)\> + +#### Defined in + +[packages/types/src/monitoring-server.ts:16](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/monitoring-server.ts#L16) diff --git a/docs/types/interfaces/IMonitoringServerConstructor.md b/docs/types/interfaces/IMonitoringServerConstructor.md new file mode 100644 index 000000000..f23f1bb4b --- /dev/null +++ b/docs/types/interfaces/IMonitoringServerConstructor.md @@ -0,0 +1,25 @@ +[@scramjet/types](../README.md) / [Exports](../modules.md) / IMonitoringServerConstructor + +# Interface: IMonitoringServerConstructor + +## Table of contents + +### Constructors + +- [constructor](IMonitoringServerConstructor.md#constructor) + +## Constructors + +### constructor + +β€’ **new IMonitoringServerConstructor**(`opts`) + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `opts` | [`MonitoringServerOptions`](../modules.md#monitoringserveroptions) | + +#### Defined in + +[packages/types/src/monitoring-server.ts:20](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/monitoring-server.ts#L20) diff --git a/docs/types/modules.md b/docs/types/modules.md index 8bb4ddfdc..8b77e9aa4 100644 --- a/docs/types/modules.md +++ b/docs/types/modules.md @@ -17,6 +17,8 @@ - [ILifeCycleAdapter](interfaces/ILifeCycleAdapter.md) - [ILifeCycleAdapterMain](interfaces/ILifeCycleAdapterMain.md) - [ILifeCycleAdapterRun](interfaces/ILifeCycleAdapterRun.md) +- [IMonitoringServer](interfaces/IMonitoringServer.md) +- [IMonitoringServerConstructor](interfaces/IMonitoringServerConstructor.md) - [IObjectLogger](interfaces/IObjectLogger.md) - [ISequenceAdapter](interfaces/ISequenceAdapter.md) - [ReadableStream](interfaces/ReadableStream.md) @@ -116,6 +118,7 @@ - [MessageDataType](modules.md#messagedatatype) - [MessageType](modules.md#messagetype) - [Middleware](modules.md#middleware) +- [ModuleLoaderOpts](modules.md#moduleloaderopts) - [MonitoringHandler](modules.md#monitoringhandler) - [MonitoringMessage](modules.md#monitoringmessage) - [MonitoringMessageCode](modules.md#monitoringmessagecode) @@ -124,6 +127,9 @@ - [MonitoringMessageHandler](modules.md#monitoringmessagehandler) - [MonitoringRateMessage](modules.md#monitoringratemessage) - [MonitoringRateMessageData](modules.md#monitoringratemessagedata) +- [MonitoringServerConfig](modules.md#monitoringserverconfig) +- [MonitoringServerOptions](modules.md#monitoringserveroptions) +- [MonitoringServerValidator](modules.md#monitoringservervalidator) - [MutatingMonitoringMessageHandler](modules.md#mutatingmonitoringmessagehandler) - [NetworkInfo](modules.md#networkinfo) - [NetworkInfoMessage](modules.md#networkinfomessage) @@ -497,7 +503,7 @@ ___ #### Defined in -[packages/types/src/sth-configuration.ts:4](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L4) +[packages/types/src/sth-configuration.ts:5](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L5) ___ @@ -514,7 +520,7 @@ ___ #### Defined in -[packages/types/src/sth-configuration.ts:16](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L16) +[packages/types/src/sth-configuration.ts:17](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L17) ___ @@ -1007,7 +1013,7 @@ Host process configuration. #### Defined in -[packages/types/src/sth-configuration.ts:41](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L41) +[packages/types/src/sth-configuration.ts:42](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L42) ___ @@ -1312,7 +1318,7 @@ ___ #### Defined in -[packages/types/src/sth-configuration.ts:75](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L75) +[packages/types/src/sth-configuration.ts:76](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L76) ___ @@ -1551,7 +1557,7 @@ ___ #### Defined in -[packages/types/src/index.ts:41](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/index.ts#L41) +[packages/types/src/index.ts:43](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/index.ts#L43) ___ @@ -1675,6 +1681,22 @@ ___ ___ +### ModuleLoaderOpts + +Ζ¬ **ModuleLoaderOpts**: `Object` + +#### Type declaration + +| Name | Type | +| :------ | :------ | +| `name` | `string` | + +#### Defined in + +[packages/types/src/module-loader.ts:1](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/module-loader.ts#L1) + +___ + ### MonitoringHandler Ζ¬ **MonitoringHandler**: (`resp`: [`MonitoringMessageFromRunnerData`](modules.md#monitoringmessagefromrunnerdata)) => `MaybePromise`<[`MonitoringMessageFromRunnerData`](modules.md#monitoringmessagefromrunnerdata)\> @@ -1812,6 +1834,52 @@ ___ ___ +### MonitoringServerConfig + +Ζ¬ **MonitoringServerConfig**: `Object` + +#### Type declaration + +| Name | Type | +| :------ | :------ | +| `host?` | `string` | +| `path?` | `string` | +| `port` | `number` | + +#### Defined in + +[packages/types/src/monitoring-server.ts:3](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/monitoring-server.ts#L3) + +___ + +### MonitoringServerOptions + +Ζ¬ **MonitoringServerOptions**: [`MonitoringServerConfig`](modules.md#monitoringserverconfig) & { `check?`: [`MonitoringServerValidator`](modules.md#monitoringservervalidator) \| [`MonitoringServerValidator`](modules.md#monitoringservervalidator)[] } + +#### Defined in + +[packages/types/src/monitoring-server.ts:11](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/monitoring-server.ts#L11) + +___ + +### MonitoringServerValidator + +Ζ¬ **MonitoringServerValidator**: () => `MaybePromise`<`boolean`\> + +#### Type declaration + +β–Έ (): `MaybePromise`<`boolean`\> + +##### Returns + +`MaybePromise`<`boolean`\> + +#### Defined in + +[packages/types/src/monitoring-server.ts:9](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/monitoring-server.ts#L9) + +___ + ### MutatingMonitoringMessageHandler Ζ¬ **MutatingMonitoringMessageHandler**<`T`\>: (`msg`: [`EncodedMessage`](modules.md#encodedmessage)<`T`\>) => `MaybePromise`<[`EncodedMessage`](modules.md#encodedmessage)<`T`\> \| ``null``\> @@ -2084,7 +2152,7 @@ PreRunner container configuration. #### Defined in -[packages/types/src/sth-configuration.ts:31](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L31) +[packages/types/src/sth-configuration.ts:32](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L32) ___ @@ -2104,7 +2172,7 @@ ___ #### Defined in -[packages/types/src/sth-configuration.ts:274](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L274) +[packages/types/src/sth-configuration.ts:277](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L277) ___ @@ -2211,7 +2279,7 @@ Runner container configuration. #### Defined in -[packages/types/src/sth-configuration.ts:36](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L36) +[packages/types/src/sth-configuration.ts:37](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L37) ___ @@ -2291,6 +2359,8 @@ ___ | `k8sSequencesRoot` | `string` | | `k8sSthPodHost` | `string` | | `logLevel` | [`LogLevel`](modules.md#loglevel) | +| `monitoringServer` | { `port`: `number` } | +| `monitoringServer.port` | `number` | | `platformApi` | `string` | | `platformApiKey` | `string` | | `platformApiVersion` | `string` | @@ -2348,6 +2418,7 @@ ___ | `kubernetes` | `Partial`<[`K8SAdapterConfiguration`](modules.md#k8sadapterconfiguration)\> | Kubernetes adapter configuration | | `logColors` | `boolean` | Enable colors in logging. | | `logLevel` | [`LogLevel`](modules.md#loglevel) | Logging level. | +| `monitorgingServer?` | [`MonitoringServerConfig`](modules.md#monitoringserverconfig) | - | | `platform?` | { `api`: `string` ; `apiKey`: `string` ; `apiVersion`: `string` ; `hostType`: ``"hub"`` \| ``"federation"`` ; `space`: `string` } | - | | `platform.api` | `string` | - | | `platform.apiKey` | `string` | - | @@ -2367,7 +2438,7 @@ ___ #### Defined in -[packages/types/src/sth-configuration.ts:109](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L109) +[packages/types/src/sth-configuration.ts:110](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/sth-configuration.ts#L110) ___ diff --git a/docs/types/modules/MRestAPI.md b/docs/types/modules/MRestAPI.md index cd1f3831a..410a60fba 100644 --- a/docs/types/modules/MRestAPI.md +++ b/docs/types/modules/MRestAPI.md @@ -287,17 +287,17 @@ ___ #### Defined in -[packages/types/src/rest-api-manager/post-disconnect.ts:3](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/rest-api-manager/post-disconnect.ts#L3) +[packages/types/src/rest-api-manager/post-disconnect.ts:4](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/rest-api-manager/post-disconnect.ts#L4) ___ ### PostDisconnectResponse -Ζ¬ **PostDisconnectResponse**: [`OpResponse`](MMRestAPI.md#opresponse)<{ `disconnected`: { `reason`: `string` ; `sthId`: `string` }[] ; `managerId`: `string` }\> +Ζ¬ **PostDisconnectResponse**: [`OpResponse`](MMRestAPI.md#opresponse)<{ `disconnected`: { `reason`: `string` ; `sthId`: `string` }[] ; `managerId`: `string` }\> \| { `error`: `string` ; `opStatus`: `ReasonPhrases` } #### Defined in -[packages/types/src/rest-api-manager/post-disconnect.ts:9](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/rest-api-manager/post-disconnect.ts#L9) +[packages/types/src/rest-api-manager/post-disconnect.ts:10](https://github.com/scramjetorg/transform-hub/blob/HEAD/packages/types/src/rest-api-manager/post-disconnect.ts#L10) ___