Feature docs (#1037)

* convert test tables in showcase to Table instances

* refine showcase Tables

* GridLayout experiments

* table loading fix, add instrument

fix intermittent bug with table loading
move add instrument to col header for basket trading

* responsive basket toolbar. on order status

docs/ui/vuu_ui_features.md

@@ -46,16 +46,26 @@ It is not ideal that all bundles must be created, together with the runtime shel
 
 ## How does a Feature manage data communication with the Vuu Server ?
 
+Data from a remote Vuu server instance is managed using one or more client side DataSource instances. A single DataSource represents a subscription to a Vuu table. A Feature may need to create more than one dataSource. The basket trading feature is an example of a complex feature that creates five dataSources. The Filter Table feature creates a single dataSource. There is a pattern for dataSource management that all features should use and that can be seen in all the existing Features provided with the sample app. Once created, a dataSource should be saved using the session storage mechanism provided by Vuu. When loading, a Feature should first try to load dataSources from session state. If not found, they can be instantiated then saved into session state. THe reason for this is that there are some runtime scenarios that will result in a React component being unmounted, then remounted. When this happens, we want to try and avoid tearing down then recreating the server subscription(s). By storing the dataSource(s) in session state , they persist across React component remount events and can be reloaded back into the original owning component. In the future, as the Feature API is evolved, we will look at baking this behaviour into `feature datasources`. Note: if users do not do this and build Features that always create new dataSource instances whenever mounted, they will work as expected, if less efficient in terms of server resources.
+
 ## How does a Vuu app know which Feature(s) to load ?
 
+The sample app provided with Vuu does not load any features by default. It initially renders only the Shell. It does provide a mechanism to allow a user to add features to the app at runtime - from the palettes available in the Left Nav. The `Vuu Features` palette offers the Basket Trading and Instrument Tiles features. The `Vuu Tables` palette hosts the Filter Table feature, which can be dragged onto the main content area of the app, using any one of the listed Vuu tables.
+In a real world application, the app would just as likely be built with one or more features built-in and preloaded by default.
+
 ## Getting started - how do I create a Feature ?
 
+TBC
+
 ## Does the Vuu Showcase support Features ?
 
 Yes it does. The Vuu showcase is a developer tool that allows components to be rendered in isolation, with hot module reloading for a convenient developer experience. Features are a little more complex to render here because of the injection of props by the Vuu Shell
 and the fact that many features will create Vuu datasources. When running in the Showcase, we might want to use local datasources with no requirement to have a Vuu server instance running. All of this can be achieved and existing Showcase examples demonstrate how this is done.
 
 ### dataSource creation/injection in Showcase features
 
-Most features will create Vuu dataSource(s) internally. However there is a pattern for dataSource creation, descibed above. Features should use the session state service provided by the Vuu shell to manage any dataSources created. The Showcase can take advantage of this pattern by pre-populating the session store with dataSources. The Feature, when loaded will then use these dataSources rather than creating.
-In the existing Showcase examples, there is a `features` folder. The components in here are wrappers round actual VuuFeatures implemented in sample-apps. These render the actual underlying feature but only after creating local versions of the dataSource(s) required by those features and storing them in session state. WHen these features are rendered in SHowcase examples, they will be using the local test data.
+Most features will create Vuu dataSource(s) internally. However there is a pattern for dataSource creation, described above, which helps us here. Features should use the session state service provided by the Vuu shell to manage any dataSources created. The Showcase can take advantage of this pattern by pre-populating the session store with one or more dataSources. The Feature, when loaded will then use these dataSources rather than creating new dataSource instances.
+In the existing Showcase examples, there is a `features` folder. The components in here are wrappers round actual VuuFeatures implemented in `sample-apps`. These render the actual underlying feature but only after creating local versions of the dataSource(s) required by those features and storing them in session state. When these features are rendered in Showcase examples, they will be using the local test data.
+
+Within the `examples` folder of Showcase, the `VuuFeatures` folder has examples of usage of the BasketTrading, InstrumentTiles and FilterTable features.
+Each of these has at least two exported examples. One exports the example feature directly, using it as a regular React component. The other exports the example using the actual `Feature` dynamic loader component, which loads the feature from a url. This mimics almost exactly the way features are loaded into a Vuu app. The actual urls employed will vary depending on whether the Showcase is being run in dev mode (with hot module reloading) or with a built version of Showcase. Both sets of urls can be seen in the examples and the set approprtate to the current environment will be used. The Showcase build is set up to define bundle examples as separate entry points so feature bundles are created.

vuu-ui/packages/vuu-data-react/src/hooks/useVuuTables.ts

@@ -1,5 +1,6 @@
-import { useCallback, useEffect, useState } from "react";
 import { getServerAPI, TableSchema } from "@finos/vuu-data";
+import { VuuTable } from "@finos/vuu-protocol-types";
+import { useCallback, useEffect, useMemo, useState } from "react";
 
 export const useVuuTables = () => {
   const [tables, setTables] = useState<Map<string, TableSchema> | undefined>();
@@ -14,6 +15,7 @@ export const useVuuTables = () => {
 
   useEffect(() => {
     async function fetchTableMetadata() {
+      console.log("GET TABLE LIST");
       const server = await getServerAPI();
       const { tables } = await server.getTableList();
       const tableSchemas = buildTables(
@@ -29,3 +31,6 @@ export const useVuuTables = () => {
 
   return tables;
 };
+
+export const getVuuTableSchema = (table: VuuTable) =>
+  getServerAPI().then((server) => server.getTableSchema(table));

vuu-ui/packages/vuu-data-test/src/Table.ts

@@ -1,39 +1,56 @@
-import { TableSchema } from "@finos/vuu-data";
-import { VuuRowDataItemType } from "@finos/vuu-protocol-types";
+import { SchemaColumn, TableSchema } from "@finos/vuu-data";
+import { VuuRowDataItemType, VuuTable } from "@finos/vuu-protocol-types";
 import { ColumnMap, EventEmitter } from "@finos/vuu-utils";
+import { UpdateGenerator } from "./rowUpdates";
 
 export type TableEvents = {
   delete: (row: VuuRowDataItemType[]) => void;
   insert: (row: VuuRowDataItemType[]) => void;
-  update: (row: VuuRowDataItemType[], columnName: string) => void;
+  update: (row: VuuRowDataItemType[], columnName?: string) => void;
 };
 
 export class Table extends EventEmitter<TableEvents> {
   #data: VuuRowDataItemType[][];
   #dataMap: ColumnMap;
   #indexOfKey: number;
   #schema: TableSchema;
+
   constructor(
     schema: TableSchema,
     data: VuuRowDataItemType[][],
-    dataMap: ColumnMap
+    dataMap: ColumnMap,
+    updateGenerator?: UpdateGenerator
   ) {
     super();
     this.#data = data;
     this.#dataMap = dataMap;
     this.#schema = schema;
     this.#indexOfKey = dataMap[schema.key];
+    updateGenerator?.setTable(this);
+    updateGenerator?.setRange({ from: 0, to: 20 });
   }
 
   get data() {
     return this.#data;
   }
 
+  get map() {
+    return this.#dataMap;
+  }
+
+  get schema() {
+    return this.#schema;
+  }
+
   insert(row: VuuRowDataItemType[]) {
     this.#data.push(row);
     this.emit("insert", row);
   }
 
+  findByKey(key: string) {
+    return this.#data.find((d) => (d[this.#indexOfKey] = key));
+  }
+
   update(key: string, columnName: string, value: VuuRowDataItemType) {
     const rowIndex = this.#data.findIndex(
       (row) => row[this.#indexOfKey] === key
@@ -47,4 +64,111 @@ export class Table extends EventEmitter<TableEvents> {
       this.emit("update", newRow, columnName);
     }
   }
+  updateRow(row: VuuRowDataItemType[]) {
+    const key = row[this.#indexOfKey];
+    const rowIndex = this.#data.findIndex(
+      (row) => row[this.#indexOfKey] === key
+    );
+    if (rowIndex !== -1) {
+      this.#data[rowIndex] = row;
+      this.emit("update", row);
+    }
+  }
 }
+
+export const buildDataColumnMap = (schema: TableSchema) =>
+  Object.values(schema.columns).reduce<ColumnMap>((map, col, index) => {
+    map[col.name] = index;
+    return map;
+  }, {});
+
+const getServerDataType = (
+  columnName: string,
+  { columns: cols1, table: t1 }: TableSchema,
+  { columns: cols2, table: t2 }: TableSchema
+) => {
+  const col1 = cols1.find((col) => col.name === columnName);
+  const col2 = cols2.find((col) => col.name === columnName);
+  if (col1 && col2) {
+    if (col1.serverDataType === col2.serverDataType) {
+      return col1.serverDataType;
+    } else {
+      throw Error(
+        `both tables ${t1.table} and ${t2.table} implement column ${columnName}, but with types differ`
+      );
+    }
+  } else if (col1) {
+    return col1.serverDataType;
+  } else if (col2) {
+    return col2.serverDataType;
+  } else {
+    throw Error(`WTF how is this possible`);
+  }
+};
+
+// Just copies source tables, then registers update listeners.
+// Not terribly efficient, but good enough for showcase
+export const joinTables = (
+  joinTable: VuuTable,
+  table1: Table,
+  table2: Table,
+  joinColumn: string
+) => {
+  const { map: m1, schema: schema1 } = table1;
+  const { map: m2, schema: schema2 } = table2;
+  const k1 = m1[joinColumn];
+  const k2 = m2[joinColumn];
+
+  const combinedColumns = new Set(
+    [...schema1.columns, ...schema2.columns].map((col) => col.name).sort()
+  );
+
+  const combinedSchema: TableSchema = {
+    key: joinColumn,
+    table: joinTable,
+    columns: Array.from(combinedColumns).map<SchemaColumn>((columnName) => ({
+      name: columnName,
+      serverDataType: getServerDataType(columnName, schema1, schema2),
+    })),
+  };
+
+  const data: VuuRowDataItemType[][] = [];
+  const combinedColumnMap = buildDataColumnMap(combinedSchema);
+  const start = performance.now();
+  for (const row of table1.data) {
+    const row2 = table2.findByKey(String(row[k1]));
+    if (row2) {
+      const out = [];
+      for (const column of table1.schema.columns) {
+        const value = row[m1[column.name]];
+        out[combinedColumnMap[column.name]] = value;
+      }
+      for (const column of table2.schema.columns) {
+        const value = row2[m2[column.name]];
+        out[combinedColumnMap[column.name]] = value;
+      }
+
+      data.push(out);
+    }
+  }
+  const end = performance.now();
+  console.log(`took ${end - start} ms to create join table ${joinTable.table}`);
+
+  const newTable = new Table(combinedSchema, data, combinedColumnMap);
+
+  table2.on("update", (row) => {
+    const keyValue = row[k2] as string;
+    const targetRow = newTable.findByKey(keyValue);
+    if (targetRow) {
+      const updatedRow = targetRow.slice();
+      for (const { name } of table2.schema.columns) {
+        if (row[m2[name]] !== updatedRow[combinedColumnMap[name]]) {
+          updatedRow[combinedColumnMap[name]] = row[m2[name]];
+        }
+      }
+      newTable.updateRow(updatedRow);
+    }
+  });
+
+  return newTable;
+};

vuu-ui/packages/vuu-data-test/src/TickingArrayDataSource.ts

@@ -17,7 +17,6 @@ import {
   VuuRowDataItemType,
 } from "@finos/vuu-protocol-types";
 import { metadataKeys } from "@finos/vuu-utils";
-import { UpdateGenerator, UpdateHandler } from "./rowUpdates";
 import { Table } from "./Table";
 
 export type RpcService = {
@@ -32,21 +31,18 @@ export interface TickingArrayDataSourceConstructorProps
   menuRpcServices?: RpcService[];
   rpcServices?: RpcService[];
   table?: Table;
-  updateGenerator?: UpdateGenerator;
 }
 
 export class TickingArrayDataSource extends ArrayDataSource {
   #menuRpcServices: RpcService[] | undefined;
   #rpcServices: RpcService[] | undefined;
-  #updateGenerator: UpdateGenerator | undefined;
   #table?: Table;
 
   constructor({
     data,
     menuRpcServices,
     rpcServices,
     table,
-    updateGenerator,
     menu,
     ...arrayDataSourceProps
   }: TickingArrayDataSourceConstructorProps) {
@@ -60,75 +56,30 @@ export class TickingArrayDataSource extends ArrayDataSource {
     this._menu = menu;
     this.#menuRpcServices = menuRpcServices;
     this.#rpcServices = rpcServices;
-    this.#updateGenerator = updateGenerator;
     this.#table = table;
-    updateGenerator?.setDataSource(this);
-    updateGenerator?.setUpdateHandler(this.processUpdates);
 
     if (table) {
       table.on("insert", this.insert);
-      table.on("update", this.update);
+      table.on("update", this.updateRow);
     }
   }
 
   async subscribe(subscribeProps: SubscribeProps, callback: SubscribeCallback) {
     const subscription = super.subscribe(subscribeProps, callback);
-    if (subscribeProps.range) {
-      this.#updateGenerator?.setRange(subscribeProps.range);
-    }
+    // if (subscribeProps.range) {
+    //   this.#updateGenerator?.setRange(subscribeProps.range);
+    // }
     return subscription;
   }
 
   set range(range: VuuRange) {
     super.range = range;
-    this.#updateGenerator?.setRange(range);
+    // this.#updateGenerator?.setRange(range);
   }
   get range() {
     return super.range;
   }
 
-  private processUpdates: UpdateHandler = (rowUpdates) => {
-    const updatedRows: DataSourceRow[] = [];
-    const data = super.currentData;
-    for (const [updateType, ...updateRecord] of rowUpdates) {
-      switch (updateType) {
-        case "U": {
-          const [rowIndex, ...updates] = updateRecord;
-          const row = data[rowIndex as number].slice() as DataSourceRow;
-          if (row) {
-            for (let i = 0; i < updates.length; i += 2) {
-              const colIdx = updates[i] as number;
-              const colVal = updates[i + 1];
-              row[colIdx] = colVal;
-            }
-            // eslint-disable-next-line @typescript-eslint/ban-ts-comment
-            // @ts-ignore
-            // TODO this is problematic if we're filtered
-            // we need to update the correct underlying row
-            data[rowIndex] = row;
-            updatedRows.push(row);
-          }
-          break;
-        }
-        case "I": {
-          this.insert(updateRecord);
-
-          break;
-        }
-        case "D": {
-          console.log(`delete row`);
-          break;
-        }
-      }
-    }
-    super._clientCallback?.({
-      clientViewportId: super.viewport,
-      mode: "update",
-      rows: updatedRows,
-      type: "viewport-update",
-    });
-  };
-
   private getSelectedRows() {
     return this.selectedRows.reduce<DataSourceRow[]>((rows, selected) => {
       if (Array.isArray(selected)) {